When Is Improvement Not? Worsening Scores after Grant Proposal Revision
Year: 2023
Type: article
Abstract: by an NIH study section.The reviews come back with a score that is great but doesn't make the pay line.A revised application is submitted that addresses the concerns that were raised about the original application.The second reviews come back . . .and the score is worse . . .or the application doesn't even get discussed.What happened?How can this be?The grant application process, which includes the ability to revise reviewed but unfunded proposals, ought to provide an opportunity to improve the quality of the proposed research since the criticisms often identify important conceptual or methodological problems with the experimental plan.When a reviewer provides constructive feedback in the comments, the applicant should be able to benefit from that wisdom.Indeed, applicants work hard to respond to the reviews, oftentimes doing more experiments to include additional preliminary data that address the concerns and further support the feasibility of what is being proposed.Hence, for most applications one can anticipate that the revision process improves the proposed work.However, such improvements are sometimes not reflected in the subsequent score.Although we can accept that for some applications the revisions introduce new experimental foibles that would worsen the score, we suspect that this occurs in a minority of revised proposals.The message that is sent when the next review says the "previous reviewers' concerns have been adequately addressed" but the score gets worse is: "We provided some advice on how to improve your application, you followed it, and now the application is not as good as the first time through."This is unreasonable.Does this mean the reviewers' comments were off the mark in the first review?That would be a serious concern.Those of us who have sat on study section have heard the instructions from the NIH: each review cycle is independent, which means that new reviewers may have new concerns or view the work to be less important.We understand that when an application gets new reviewers, these could identify flaws that were not apparent in the first review, but when this happens it should be clearly stated.Today, reviewers are asked to comment on the quality of the revision, but they are not privy to the final score during the voting process, which is averaged after the meeting and known only to the study section scientific review officer.In those rare cases where the review of a revised proposal identifies a problem that was missed on the first review, a justification for a worse score should be specifically noted in the critique.This would help an applicant who receives a worse score after revision know what happened and maintain the credibility of the system.When a grant proposal is reviewed by study section, the scores and the comments represent the consensus of the group, not simply the thoughts of the assigned reviewers.We know this because every nonconflicted member of the study section has the opportunity to discuss the proposal, to suggest edits to the comments, and to assign a score.Indeed, members are allowed to "vote outside the range" of what is recommended by the reviewers provided that they disclose their intentions, meaning the reviewers' comments are not the be all and end all.The summary statement and the score represent the group
Source: mSphere
Authors: Michael J. Imperiale, Arturo Casadevall
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $3,500
Funder: National Institutes of Health
OpenAlex ID: W4363643591