New Scoring System for Proposals Submitted in 2025
Why?
The NIH announced that proposals submitted after January 2025 will be scored using a new system. Why, you may be asking yourself, are they changing a system that worked pretty well? I watched their webinar so you don’t have to, and they listed three reasons for the changes.
Simplify the system to help ‘peer reviewers’ focus reviews
Mitigate effects of reputational bias
Reduce reviewer burden
A bit more explanation: The first point, simplification, was to align scores around 3 central questions–1) Should the work be done (importance); 2) Can the work be done (rigor and reproducibility) and 3) Will the work get done (environment, team and resources). These seem like positive changes that could simplify and clarify the reviews, thus increasing quality of reviews returned.
The second point, minimizing reputational bias, is supposed to be accomplished by reducing the summary of the expertise of the team and the team’s ability to get the work done to a binary ‘appropriate/additional resources’ required. Reviewers will have the option to explain where they think the environment and resources fall short.
The new set up seems likely to reduce reviewer burden, by making the environment and resources score binary and because now all assessments not related directly to scientific merit, such as Resource Sharing sections, will shift to NIH staff, rather than reviewers.
What do these changes mean for proposal strategy?
I see two areas where PIs will have to make some changes to proposal strategy. The first is related to simplifying the system to help reviewers focus feedback. With this goal, the new scoring system collapses Significance and Innovation into one score, Importance, ranked from 1-9. Yikes. I think this will cause some issues. Under the old system, it was possible to get good scores with a not-terribly-innovative proposal. You could score very well with a low-ish Innovation score, if your Significance section and Approach were rock solid. The majority of proposals I see actually fall into this category because a lot of very good work may not be that innovative because it is improving on previous work or advancing the field in one specific but incremental way. Now, work that falls into this category might have a harder time because these categories are lumped into one score and (as far as I could tell) the new framework does not specify exactly how much weight reviewers should place on Innovation when deciding on the single score. I will keep searching for guidance on this question, as NIH’s website says reviewer guidance will be released this summer. So far, I have not seen any.
I do not know whether the NIH chose this new scoring framework because they actively want to discourage work that is not hugely innovative, or this potential downgrading of incremental research is an unintended consequence. Either way, I am not a fan of this change because, although innovation is great and important, a lot of necessary and impactful research may not be hugely innovative. Alas, I was not consulted in devising this new framework! In my opinion, a single score may overweight importance of innovation.
The second change that calls for revisions to grants(person)ship is the binary score for resources and environment, which includes investigators. It will be essential to not get shunted into the ‘additional resources needed’ category because the nature of binary scores and human brains is that when you get put into the ‘not good enough’ box, even if there are qualifiers added, a negative cloud hangs over the work. Therefore, additional thought, time, and effort should be spent getting the reviewer to a place where s/he can select ‘adequate’ for this category. That means making sure your papers get out and published in a timely fashion, adding consultants or Co-Is to cover your areas of weakness, and making sure everyone on the proposal has a key role and is actively involved. I will devote another newsletter to this topic in future.
This webinar lists all the NOFOs that will be affected by the new framework, about 2 mins 30 seconds in.
I hope everyone is enjoying summer and please reach out with thoughts, questions, or to update me on your funding outlook and recent successes or non-successes (let’s not use the F word) at my email, [email protected].
CONTACT US
follow on Twitter | follow on LinkedIn| forward to a friend