About 6 weeks ago, I was invited to write an editorial for Ideas in Ecology and Evolution (IEE). I recalled reading a few articles by Hochberg commenting on the increasing difficulty in securing referees and had seen several new incentive systems including PubCreds discussed by oikosjeremy. So, I am just posting it here, not to promote it, but to provide a forum for discussion and synthesis on the topic.
The editorial summarizing the outcome of a poll I conducted on the ecolog listserve and my musing is online, open-access here (just scroll down a bit on that first page and you will see the current articles). Click on the money for nothing one then on pdf and bingo.
http://library.queensu.ca/ojs/index.php/IEE/index
I would love to hear what the Oikos and Nordic Ecological Society thinks about this topic. I did this mainly as an exercise for us to push for more transparency in peer review in general (including the incentives and profit) and for us to consider whether the model we use to handle papers could be improved. Maybe the new EiC for Oikos will solve this. I would also love to hear from the Oikos editors and referees to see if we would ever consider this.
cheers,
chris.
The idea of paying referees has been discussed in various venues. I exchanged some ideas with fellow ecological blogger Ethan White a while back. See http://www.ipetitions.com/petition/fix-peer-review/blog/2680.
Basically, the reason Owen Petchey and I thought up PubCreds was to get around various problems with paying peer reviewers with real money. One can of course argue about whether PubCreds has its own problems which are even more serious.
By: oikosjeremy on December 30, 2011
at 9:09 pm
Agreed. Cool idea. It would be nice to be direct though. What are your thoughts on this now? Free page charges or colour reproductions next time you publish with a journal that you review for a lot (on time)?
Or fast-tracked reviews of your own work? Like a coffee shop stamp card. Do 10 reviews, next paper submitted gets faster handling (no difference in outcome, just editors make a point to get it to referees much quicker).
By: oikoschris on December 30, 2011
at 9:17 pm
The thing about those sorts of incentives is that they are hard to callibrate, and people place very different marginal values on them. A virtue of pubcreds is that it directly links your reviewing and submitting.
By: oikosjeremy on December 31, 2011
at 12:37 am
what do you think is the best solution? I was thinking something like a coffee shop stamp card with each journal you review for. Review a lot, on time, and get free page charges or colour reproductions.
By: Anonymous on December 30, 2011
at 9:14 pm
I think it’s a discussion that’s well worth having, but as Jeremy points out, there’s usually a lot more to consider once you start getting under the surface of the discussion.
If you’re talking purely financial recompense, one has to ask where the dough comes from? For-profit publishing houses might not be so keen to shell out cash to reviewers (even though their profits rely on our work). Society run journals may not even have money to spend on these things. Journals offering “fast-track” reviews can too easily be let down by reviewers, without offering extra incentives to entice reviewers to submit rapidly. And it’s arguable that we should be spending more time reviewing articles to improve quality, not less.
The PubCred and related ideas get round that quite easily by dealing with other forms of recompense, but (as Jeremy also points out) there are other issues that need to be dealt with here, such as the “guns-for-hire” question of who deserves an authorship on a paper.
The Scholarly Kitchen blog touches on many of these and related points (if you dig through their archives), as well as linking to those few academic studies that have actually tried to quantify the size of the reviewing problem – if it even exists. The evidence (scientific, rather than anecdotal) seems to suggest that the problem isn’t so big after all – in certain fields. This doesn’t include the fascinating suggestion that Quality reviewing declines with experience though.
Personally, I think it would be great to have more academic credit for the reviewing work we do, as well as more transparency surrounding all of these issues. Plenty of interesting topics for discussion!
By: Mike Fowler on December 31, 2011
at 11:16 am
Thanks Mike. I will take a look at those sites. Academic credit is a nice distinction and really important. I am also familiar with the quality issue and the research on it too. Surprising.
I was also musing on credit in that most academics have merit awards, tenure hoops to hop through, etc. so credit in the form of recognition would be nice to be able to bring back to one’s employer (if needed/desired). A standard form letter drafted by the editors that they provide at certain benchmark review levels could be cool. Like a frequent flyer rewards program card. You could frame it and put up on the wall even. ha.
The other limitation I was considering was that you may review a lot with one journal but publish with another. So, society level credit would be nice transcending the specific journal. The other issue might be that you might not get asked frequently enough to earn credit. I suspect some established folks get many many requests whereas others get less. So, that is a bit tricky too.
With respect to the crisis, that is the million dollar question! I would love to know how many individuals editors have to ask to get just two!!
By: oikoschris on December 31, 2011
at 4:14 pm
[...] on from previous posts on reforming peer review (see here, here, and here), I wanted to note a new peer review service, Peerage of Science. PoS is a private [...]
By: Another peer review reform: Peerage of Science « Oikos Blog on January 23, 2012
at 5:40 am
Dear all,
Put it like that: I mainly due reviews of papers during weekends or late in the evening because I am busy doing research, teaching, and administration for my institute during work hours. So, I am either getting less sleep or less time with my family to keep the system working. But the system is not run by colleagues. It is run by private corporations that make substantial profit out of a highly subsidized business and free work from a specialized labour force. So, I am sorry, but I don’t see a reason why referes should not be financially compensated for their work.
The publishing business has to internalize its costs if it wants to be sustainable. Paying referees (and editors) is part of the cost of running such a business. Exceptions can be made for non-profit organizations, like professional societies, since members may want to contribute with their work for the development of the society. But I don’t see a reason why such logic would apply to the highly profitable business of Wiley, Elsevier, or Springer.
Miguel
By: Miguel Bastos Araújo on January 27, 2012
at 10:40 am