All papers published in Nature (but not baby Nature journals, yet) will be published alongside the referee reports and author rebuttals.
Some of the given motivations are increased transparency and trust in the scientific method, as well as giving early career researchers (who may lack many opportunities to see reports and review) a chance to see inside the process.
We should keep in mind that publishing reports and rebuttals is not the whole story. The identities of the anonymous referees are also important:
- Referee A gave a highly scathing report, but recently uploaded a competing manuscript on arXiv.
- Referee B was very positive, but they were grad students in the same group as the corresponding author many years ago and are still friends, even though they do not collaborate.
- Referee C gave a very brief report that seems not very well thought out, but they are a giant in the field anticipated to receive a Nobel Prize someday.
- Referee D has reviewed dozens of papers for the journal and recommends rejection 95% of the time.
Missing this context, some editorial decisions will seem confusing to the outside reader. Referee C's report might seem unprofessional, but it carries the weight of decades of experience.
There is also an implicit selection bias - only papers that make it to publication will have reports published. This excludes papers that receive negative reviews, and papers that are desk rejected by the editors.
All these limitations mean that open peer review needs to be complemented with mentorship by more experienced researchers.
Unfortunately, a minority of experienced referees do the majority of the work. Some authors may publish prolifically in respected journals but refuse to do any reviews for them. There is a need for better incentives for referees beyond the less tangible benefits of seeing research before it is published as well as the other referee reports and author rebuttals.
There are many calls to pay referees for their service. Why not? Some grant agencies pay referees for proposals. It makes sense to pay for quality reviews when a lot of money is on the line. It is hard to devote similar resources to papers without substantially increasing the cost of publishing, not just to cover the review fees but all the associated admin expenses with paying people all over the world. This would unfairly impact less well-funded groups and referees from certain countries who may be prevented from receiving payments.
It is much more practical to offer non-monetary incentives. Previously, Optica allowed you to cash in points earned by submitting quality referee reports to get a rebate on their membership fees. I found that an effective incentive, motivating me to review a few papers a year alongside my editorial duties at APS. Unfortunately it appears to be discontinued now, perhaps because the scheme was too expensive for them. Related schemes mainly offered by for-profit publishers (publication fee discounts based on reviewing activity) are too small to be effective, especially when authors generally do not pay these fees out of their own pocket.
What other non-monetary compensation can journals offer?
One idea that was floated at the recent PRA editorial board meeting was to offer a "Second Chance Voucher" for accumulating a sufficient number of points for good reports. This voucher would allow authors to request another referee opinion on a manuscript up for rejection (either by the editors, or after review). At selective journals such as PRL or PRX, in the event of split referee recommendations it is common to reject the paper. This would give authors who give back to the community by refereeing regularly the option to get another chance. And despite the need to consult more referees on some papers, it would overall increase the pool of active referees motivated to submit quality reports.
What other non-monetary compensation would motivate you to review more papers? Comments are welcome!
No comments:
Post a Comment