NSF Proposal Merit Review Criterion 3: What constitutes a “mechanism to assess success?”
by ScienceDocs NSF Proposal Expert Dr. Laura W.
When a reviewer writes up an evaluation of your NSF proposal, they must directly address the five merit review criteria, including the third one: “Is the plan for carrying out the proposed activities well-reasoned, well-organized, and based on a sound rationale? Does the plan incorporate a mechanism to assess success?” I’ve reviewed dozens and dozens of NSF proposals, and the vast majority are strong on the first part of that criterion. Many, however, do not mention any mechanism to assess success, so the answer to that second question is “nope.” Your chances of funding go up when your reviewers can write instead, “Yes, the proponents were thoughtful about what success will look like and provided an explicit description of how they will measure it. They also mentioned what they can still learn even if the riskiest part of the project does not unfold smoothly.”
In your mind, success may look like getting five papers past Reviewer #2 and having the grad students finish their theses without too much drama. Those are wonderful accomplishments, but the NSF is looking for something else here. Reviewers and Program Officers want to know that you know how your efforts will translate into useful knowledge. Presumably, you’ll end up with maps or images or numbers in tables that can be plotted this way or that way, but what will you need to see in the data to evaluate your hypotheses meaningfully? And, if you don’t see that, what will you be able to say of value? Be explicit, specific, and as quantitative as possible. An example might be, “Our experiment is designed to have enough replicates to determine whether A and B correlate with a high confidence interval (³95%). If we succeed, we will conclude that our proposed reaction mechanism is indeed a better explanation of all the published observations than the mechanism described by Them et al. (2021). If the confidence interval is less than 95%, but higher than 80%, we will know that building an apparatus to operate at higher pressures for this experiment would be well justified in the future, despite the practical challenges.” Note that this sentence acknowledges the possibility that an unambiguous, paradigm-changing conclusion may not be possible but that the best next steps will become known. I sometimes find it difficult to think up assessment ideas for project success myself, but doing so usually makes my research plan stronger and enables reviewers to respond positively on all of Criterion 3.
Your Broader Impacts components in your NSF Proposal will be evaluated in the same way, so you need a plan for assessing that work, too. If you plan to visit 3rd grade classrooms and conduct an activity with students four times, how will you know whether your activity had the impact you had in mind? Is it enough if the teachers smile and thank you? Is there some small feedback mechanism you could design to find out whether the students were engaged and came away with new knowledge or enthusiasm about science? If you’re planning to establish a partnership between academia and industry, what would serve as sufficient evidence that the partnership is doing what motivated its creation or is greater than the sum of its parts?
As you write about your mechanism for assessing success, keep in mind that specific, pragmatic remarks about what success will look like are almost always better than nothing and better than unrealistic, sweeping statements about improving the human condition through one 3-year project. Your job is simply to demonstrate to reviewers that you have been thoughtful and clear-headed about what your data set is likely to look like, how your Broader Impacts activities will go, and how you will know to what extent the project goals were achieved.
Should you need help with your NSF Proposal, ScienceDocs can provide you with any level of support you may need. To get started, fill out a quote sheet!
