Posted by: Jeremy Fox | May 16, 2011

Should granting agencies fund projects or people?

Basic research grants, awarded to individual investigators or small teams of investigators in order to pursue specific  projects, are increasingly difficult to obtain. Applications have been increasing and success rates decreasing for years at the US National Science Foundation (NSF), US National Institutes of Health (NIH), UK Natural Environment Research Council (NERC), and other government funding agencies. Success rates at many agencies are now below 10%. In response, applicants submit more and more applications in the hopes of getting at least one funded, and repeatedly revise and resubmit the same project. This creates a significant time burden on peer reviewers, as well as on the applicants themselves. Low success rates also discourage development of long-term or risky research programs by individual investigators.

The easy ‘solution’ is to call for more funding, but in the current economic climate that is unrealistic. More importantly, as NIH discovered a few years ago (when their funding was rapidly doubled), more funding simply causes more investigators and institutions to chase that money, so that per-capita success rates end up no higher than they were before.

I wonder how many of my colleagues are aware that there’s an alternative. Canada’s Natural Science and Engineering Research Council is the Canadian equivalent of NSF or NERC, funding basic research by individual investigators in non-biomedical fields. But NSERC’s strategy is, as far as I know, unique among major government funding agencies. Put briefly, NSERC funds people, not projects. For details see here, but the salient points are as follows:

  • NSERC Discovery Grants are (typically) 5 year awards to individual investigators to fund their research programs (as opposed to individual projects within those programs). The goal is to fund excellent research programs in a long-term, sustainable way.
  • Applications are 5 pages long (not counting bibliography, budget, etc.). The emphasis is not on the technical details of specific studies, and for this reason preliminary data are rarely included. Rather, the applicant describes and motivates the long-term goals of their research program, summarizes progress over the previous 6 years, and (in the remaining 4 pages or so) lays out their plans for the next 5 years of their program.
  • There’s no expectation that you’ll do whatever you said you’d do in your previous grant. In evaluating your renewal application, NSERC evaluates the science you’ve done on its merits; they don’t care whether or not it’s the science you proposed to do 5 years ago. You need to propose a plan for the next 5 years not because NSERC expects you to necessarily follow through on it, but because if you don’t have any plan at all the odds that you’re going to do good science over the next 5 years are slim, even if your track record is good.
  • You can only hold one Discovery Grant at a time (because by definition, you only have one research program).
  • Success rates are about 60% (a bit lower for new investigators), which is actually down from 80% 10 years ago. This is in part because 1/3 of the evaluation relies on your track record (publications and other outputs over the previous 6 years). If you do good science, and you propose a reasonable plan for the next 5 years, you’re going to get renewed. You might get cut (or increased!) substantially over your previous grant, but frankly you’re not going to get cut off entirely unless you deserve it. High renewal rates are necessary if your goal is to fund long-term research programs in a sustainable way (unless you think giving out 25 year grants with very low success rates would be a good idea)
  • Keeping success rates high means keeping the average grant size relatively small–about $35,000/year. The very best people can get up to about $140,000/year, and the minimum is roughly $15,000/year.
  • There are also some technical differences between NSERC grants and grants issued by agencies in other countries, which I’ll note just for completeness. Canadian academic appointments are 12-month appointments, so applicants don’t need to apply for (and NSERC won’t pay) summer salary for themselves. Nor do applicants need to budget for overhead; overhead is handled separately. So the difference in size between a typical NSERC grant and, say, an NSF grant is less than it might seem at first glance, since no NSERC money is going to the investigator’s summer salary or to overhead

As a Canadian academic, I love the NSERC system. I love being able to sustain a baseline level of funding for my lab by writing one 5-page grant every 5 years. I love not having to review repeated revisions of other people’s grants. I love that the system cares about my track record, which considerably reduces the stochasticity of decision making. If a single referee misunderstands or doesn’t appreciate your NSF application, you’re doomed. But if you have a track record of publishing regularly in good journals, no referee is going to fail to appreciate that. I love that the system doesn’t bog everyone down in picky technical details.

It’s no problem that I can’t afford to pay a grad student’s entire salary, because Canadian grad students are mostly funded by TAships, scholarships, and fellowships during the academic year. I only have to pay them a bit of summer salary. Even so, I still can’t afford more than about 3 students on my current grant, but that’s ok because that’s my ideal lab size.

Are there features of the system I don’t love? Sure. I wish I had enough money to pay a postdoc or a full-time technician on my NSERC grant, but I don’t (I’m not one of the ‘high fliers’ whose NSERC is worth >$100K/year). I’d like to get into experimental evolution, but I probably couldn’t afford to do so on my current NSERC because gene sequencing is still too expensive. But for me, these drawbacks are trifles compared to the advantages.

I wonder how low success rates at NSF, NERC, and other agencies will have to drop before the Canadian system starts to look good to my colleagues in other countries. I’ve actually asked numerous colleagues this question, and answers have ranged from “The Canadian system already looks good to me!” to “They can never drop that low.” The latter view was expressed by a colleague who likened an NSF grant to a hit of crack cocaine–it’s so big that you’ll do anything, and put up with anything, to get another hit!

Conversely, a Canadian colleague once suggested to me that NSERC should start a new program of big, NSF-style project-based grants, funded by cutting everyone’s Discovery Grant by 5-10%. His theory was that we’re all cocky enough to think that we’d be one of the rare few who got one of these big new grants. He’s wrong about that–I’m cocky, but not that cocky. More precisely, I’m sufficiently risk-averse that I’m not willing to give up 10% of my NSERC for what’s effectively a lottery ticket. I think, in the long run, I can do more and better science in a system where my expected grant size, and the variance around that expectation, are both modest, than in a system in which the mean might (or might not!) be higher, but the variance is also higher (with a huge mode at zero).

Who’s with me?

About these ads

Responses

  1. Thanks Jeremy for an interesting post.

    The potential good side of funding people rather than projects would allow people time and possibilities to ‘think out of the box’ an not only to do the research that have been ‘ordered’ by funding agencies. I would vote for that. The risk, however, would perhaps be that a person being funded might lean back and just go with the flow, and not really do much at all. That is a risk we have to take.

    I am not at all sure that huge, for instance Pan-European projects, are the solution to all and everything. Good people need to have the time and atmosphere to sit down and think a bit, not only doing more and more research.

  2. I agree with linusoikos’ point about out-of-the-box thinking and want to elaborate on it a bit more. Not only do NIH-funded labs spend a lot of time writing and revising new proposals, they technically aren’t allowed to use existing funding to create new proposals (including the collection of pilot data that are usually necessary to get funding). This leaves labs with a bunch of unsavory options: (A) work only on what they are currently funded to do, insuring that they never explore interesting new directions; (B) lie about how they spend their time and money; (C) spend their days working on currently funded research and their nights working on new ideas, leaving no time for family, recreation, etc. The NSERC approach seems more realistic and humane, giving labs more leeway in figuring out how to use their funding.

  3. I think the Canadian system sounds awsome and have heard other scientists in Canada speaking warmly of it. In Sweden the trend is more money to bigger projects, leaving less to the rest of us. And we have nothing close to tenure. 50-year old professors still relying on soft moeny (external grants) are very common. Very sad.

  4. It is interesting however, that some Canadian scientists who have been nurtured by the NSERC system come to the US thinking that because of their publication history, they will easily do wonderfully and get the major grants and become superstars. And then they find they are totally unsuccessful because they have no clue about writing a competitive grant. Oh well, of course it is hard to generalize, but these situations do exist.

    • I’m unclear why this possibility has any implications for the merits of one funding system vs. the other. Especially since it cuts both ways–US academics who move to Canada sometimes write grants with too much methodological detail.

      It’s also worth emphasizing that your track record is only 1/3 of your NSERC Discovery grant score. It matters, but it’s not eveything. If you think your publication record alone will get you a big grant, you’ll be disappointed, in Canada as well as in the US.

      • Do you really think the faculty who write grants with too much methodological detail are going to get funded in the US? Having served on many panels, I can say – certainly not! Perhaps its the ones here who do not know how to compete, that move to Canada, to be ensured of keeping a research program? Writing a proposal in a highly competitive environment, is the best way to hone scientific skills and think deeply, provide innovation, and think of broader impacts (as NSF requires). Of course, one wishes that the success rates in the US were higher. But I have yet to know of a high quality researcher in the US who has had to shut down their research because of the lack of funding. Passion for research always succeeds, wherever.

      • You appear to have misunderstood my remark about methodological detail. NSF grants are 15 page long descriptions of a single project (which of course typically comprises a series of tightly interlinked experiments or other studies). NSERC Discovery grants are 5-page long summaries of recent progress of a long-term research program plus a plan for the next 5 years of that research program, and so necessarily include much less detail. Someone who is used to providing an optimal level of detail for an NSF grant may well struggle to adjust to writing NSERC Discovery grants, just as someone who is used to writing NSERC Discovery grants might well struggle to adjust to writing NSF grants. This is merely a specific instance of the general principle that, if you’re used to doing things one way, you may struggle to do them a different way. That was the only point I was trying to make in response to your first comment. I wouldn’t have thought that would be a controversial point.

        I look forward to being provided with links to the evidence for your claim that Canada is a haven for academics who couldn’t cut it in the US. Or do you think that I, Ben Bolker (leading computational ecologist who recently moved to McMaster), Jon Shurin (leading US trained community ecologist, at UBC for many years before recently moving back to the US), Diane Srivastava (UBC, but trained in Britain, where funding success rates are even lower than in the US), Brian McGill (leading macroecologist, US trained and at McGill U. for many years before moving to the US) and many, many others are or were in Canada because they couldn’t hack it in the US? And that’s before I’ve even listed the scads of terrific Canadian ecologists and evolutionary biologists who clearly would have no trouble hacking it in the US, although they’ve never had occasion to try (Dolph Schluter, Rees Kassen, Mark Vellend, Graham Bell…) See, that’s the problem with anecdotes–everybody can cite ones that support their own views.

        As for the notion that no “high quality” US researcher has had to shut down their research because of lack of funding, maybe not “shut down”, but certainly “scale WAY back”. That happens all the time, for instance to my own PhD supervisor, who hasn’t had an NSF grant for over a decade (forewarning: please don’t reply by trying to argue that my PhD supervisor isn’t a “high quality” researcher). And it’s going to start happening more often to NSF-funded ecologists under the NSF DEB’s new application rules, which are explicitly designed to cut down on the number of applications. Again, I’m sure you have anecdotes to support your own views–but I have mine, and so if you want to convince me you’re going to need to show some data.

        As for the idea that the US system produces better science because of the keen competition to get funded, I would merely note that it also funds short-term, incremental science rather than risky, high-impact science. And I’m not relying on anecdote for that claim; please read this post and the article linked therein for hard data on what sort of science gets funded by Canadian-type systems vs. US-type systems.

        Each system certainly has its virtues. I am not making blanket claims for the superiority of the Canadian system, or the inferiority of the US system. Nor am I seeking to air all possible arguments that might be brought to bear on the relative virtues of different systems. All I am saying is that the arguments you have provided so far for the blanket superiority of the US system are either implausible on their face, or else are not supported by evidence you have offered so far.

  5. [...] good to US investigators tired of having their careers depend on a lottery ticket? See here and here for previous Oikos blog posts on the strengths of the Canadian system relative to the US system. And [...]

  6. [...] that just won't work in the US. If you need a primer on the Canadian system, a good one is here, but the summary is as follows: The NSF equivalent, NSERC, maintains 60% success rates (currently); [...]

  7. [...] Writing in Nature, John Ioannidis shows that I’m not the only one who believes it’s time to rethink how most granting agencies fund science. [...]

  8. [...] NSERC typically provides $35,000/year, far less than the big awards from NIH. Canadian scientists love their system, valuing the stability it provides more than the possibility of large awards. Quality [...]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

Follow

Get every new post delivered to your Inbox.

Join 4,077 other followers

%d bloggers like this: