How to improve sustainability of biomedical research?

As some of you may know, NIH recently posted a Request for Information (RFI) on optimizing funding policies and other strategies to improve impact and sustainability of biomedical research. They want us (anybody, really) to submit our thoughts on the subject, limited to 500 words per question. Here it the link and here is the ScienceCareers blog post about it. The deadline for submission is May 17.

I just submitted my response. And I urge all biomedical researchers in the US, especially all postdocs and newly minted assistant professors who have not yet forgotten the postdoc life to submit their thoughts to NIH. This is our chance to change the broken system! If you don’t know what to say, it’s a good time to start thinking about these issues. To get you started, below is my response, and here is another one by Vaibhav Pai and one more from Jessica Polka, both from the FOR gang.

Comment 1
Key issues that currently limit the impact of NIH’s funding for biomedical research and challenge the sustainability of the biomedical research enterprise. We welcome responses that explain why these issues are of high importance.

Hypercompetition for faculty positions and funding is the key issue that limits the immediate impact and the long-term potential of biomedical research in the US. Here are some reasons why hypercompetition (as opposed to healthy competition) is bad for science.

  1. Hypercompetition leads to wasteful spending. First, it shifts researchers’ focus from doing science to writing grant and job applications. So, less science gets done per dollar spent. Second, a large fraction of NIH grant money that pays for postdoc training is wasted because most postdocs never make it to a PI position where they can actually apply their acquired research skills (there are far fewer PI positions than there are postdocs), and most jobs outside of academia do not require a postdoc training.
  1. Hypercompetition provides an incentive to “cut corners” in research: rather than doing careful science, there is a strong temptation to write beautiful but perhaps not well supported stories that get published in high-profile journals. This leads to the overall decline in the quality and reproducibility of science, which has been recently documented.
  1. Hypercompetition makes the profession of a scientist unattractive to young people and therefore leads to loss of talent to other professions. Why would a 20-year old want to embark on a 10-year journey (e.g., 5 years of PhD and 5 years of postdoc) whose “success” (i.e., getting a tenure-track job) depends on a chance event of a high-profile paper? Even if “success” is achieved, why would one want to be a PI in the first place, given that PI’s main job nowadays is writing grants and “packaging” the lab’s results rather than doing the actual science?

Hypercompetition, at least for faculty positions, ultimately results from the structure of the current funding system, via two mechanisms. First, postdocs are the preferred type of workforce because they cost the same or less and require shorter funding commitment on PI’s part than staff scientists or even PhD students. Thus, NIH indirectly stimulates expansion of the postdoc pool by failing to impose regulations on the working conditions of personnel hired with grant money. The second mechanism is the reward structure for PIs. PI’s success (i.e., the chance of getting funded in the next round) is his/her publication track record. Roughly speaking, the more papers the PI has recently published and the higher the impact of these papers is, the more likely is the PI to get the next grant. This reward system clearly favors bigger labs. The more people there are in your lab, the more papers your lab will produce and the more likely is at least one of these papers to be of high impact. This leads to lab-size inflation, mostly with postdocs, which in turn contributes to the unhealthy growth of the postdoc pool. It also most likely leads to a decrease in the diversity of research directions that NIH-funded investigators pursue, which limits the chances of unexpected discoveries.

Comment 2
Ideas about adjusting current funding policies to ensure both continued impact and sustainability of the NIH-supported research enterprise.  We welcome responses that point to specific strengths or weaknesses in current policies and suggest how we can build on or improve them. 

I envision a three-pronged conceptual solution to the problem of hypercompetition, at least at the level of postdocs. The key idea is to shrink and control the pool of postdocs. Rather than having a large pool of post-docs, i.e., individuals with PhDs without a defined achievable objective, there should be a small pool of pre-prof, i.e., individuals on their way to become professors. Such preprofs (often also referred to as “superpostdocs”) should form an elite pool of the best and brightest PhDs who want to take additional semi-independent training with the eventual goal of becoming a professor (similar to Bauer fellows at Harvard). The size of this pool should not exceed a factor of 4-5 of the number of annually opened biomedical tenure-track faculty position in, say, top 100 schools in the US. These pre-profs would no longer be the primary research workforce. Instead, most of the actual research should be done by graduate students and staff scientists. The pool of staff scientists would be formed by research-driven PhDs who for one reason or another did not make it to the preprof pool and preprofs who for one reason or another did not become PIs. Specific suggestions are as follows.

  1. Since a staff scientist by definition is a much more secure and stable position than a postdoc, staff scientists should not be directly funded by individual PI’s grants, which are too unstable. Instead direct costs of research grants should only go to the PI salary, equipment, and lab operation costs. On the other hand, a significant portion of grant indirect costs should be channeled towards paying staff scientist salaries and benefits. In other words, universities, not PIs, should employ staff scientists. There are obvious technical challenges that need to be resolved. For example, should PIs who bring in more money be entitled to more staff scientist slots? How should graduate students be paid? etc. But I believe these problems can be solved if appropriate committees are challenged with the task.
  1. PI success should be evaluated on the basis of efficiency, not absolute productivity. Whatever the metric of absolute productivity is, it should be normalized by the number of man-hours invested into the product. For example, rather than counting the number of papers that came out of a lab during the reporting period, one ought to, roughly speaking, divide this number by the number of man-hours available in the lab during the same period. While this policy may actually not be necessary in the long run when the postdoc pool is under control, in the short run it would curb the postdoc-pool inflation by shrinking huge labs with unacceptably high postdoc attrition rates. Again, I believe that the technical problem of devising the appropriate metrics of efficiency is solvable.

Comment 3
Ideas for new policies, strategies, and other approaches that would increase the impact and sustainability of NIH-funded biomedical research.

Funding of PhD-qualified personnel (i.e., postdocs) through direct costs of research grants should be prohibited. Instead, staff scientists should be funded by universities (through indirect costs) while 100% of superpostdocs/preprofs should be funded through institutional or individual training grants, such as K99 or programs similar to the Bauer fellow program at Harvard. This would allow NIH to have complete control of the superpostdoc/preprof pool.

Advertisements

5 responses

  1. Hi Sergei, Interesting thoughts!
    I have to say that I somewhat disagree with the notion that staff scientists are the solution. From what I’ve seen, this will be a difficult position to fill for a number of reasons.

  2. Hi Sergei,

    I understand what you mean by hyper-competition and agree with the general sentiment that you eloquently articulate. However, I see no rigorous way to define hyper-competition and distinguish it from intense competition selecting for the best. In my mind, it is all about the metrics. It is the metrics that make all the difference between competition driving better research or undermining the research enterprise. It is more fruitful to focus on metrics since most people dislikes unfairness and most people support fair competition.

    I like your idea of putting the funding in the denominator, not in the numerator as it currently is, in evaluating research outcomes.

    Perhaps a simpler and easier to implement version of your “staff scientists” proposal is to increase the minimal postdoc salary.

    Of course my favorite suggestion is to reinforce a limit on the funding per PI. NIH should rigorously limit the funding of labs in which PIs are responsible for more graduate students and post-doctoral fellows than they have time to train. PIs managing smaller labs are likely to need less money, have more time to think creatively and conceptually, provide better training to their students, and graduate fewer (and better prepared students) aspiring for academic positions.

  3. Excellent points, Sergey! Judging based on efficiency, and more staff scientists are both clear improvements to the status quo. This would help to wean us from the current pyramid scheme that demands either exponential growth or exploiting the young.

    I have an additional suggestion: give extramural scientists the option of intramural-style funding, meaning, stable, single-source funding based on retrospective review. As far as I can see, intramural NIHers (and a very few other lucky scientists) spend orders of magnitude less time chasing money than extramural PIs. I think this is partially because we never have to answer the question “what do you plan to invent and discover?”, and partially because we aren’t expected to chase multiple overlapping grants with high rejection rates.

    I’ve asked numerous decision makers here at the NIH a simple question: “How much time do your grantees spend chasing money, and how much time do they spend doing science?”. I’m ashamed to say, not one of them knew the answer. Of course, I suspect we all know the answer, and it is depressing (90/10? 95/5?). Also, I couldn’t get anyone to name even a single advantage of the extramural funding model over the intramural model.

    Note that I’m not proposing more (or less) money for extramural scientists, simply an optional different process to show you deserve it. Every five years, list your accomplishments. If they’re good, your funding improves, if they’re bad, it degrades, but it always changes gradually, and always based on the past, not based on lies and fantasies about the future.

  4. I love your contribution. Actually regarding your point 2), your are proposing nothing less than establishing a “group factor” to judge the intrinsic value of a lab. Maybe that will lead to the same seismic shift that accompanied the change of journal metrics, from total citations (which can be gamed by publishing lots of papers) to impact factor (which can be gamed, but much more difficultly)

  5. Pingback: Hiring postdocs: the PI perspective | Thoughts For Breakfast

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s