On Writing Reviews
Tags: musings, research
Reviewing papers is a central source of diversion, confusion, joy, and other feelings for many academics. Much has already been written about how to structure a review, how to phrase criticism kindly, etc. but in this brief post I want to point out that the mindset behind a review is absolutely crucial for writing a useful and constructive review. My core philosophy can be summarised as follows:
When reviewing a paper, you are primarily a midwife for the paper, not a gatekeeper.
Let me expand on this: it seems that many people consider their review duties to be akin to that of the Night’s Watch. But this is altogether the wrong approach for me! As reviewers, we are not the last line of defence that protects our community from ‘improper’ research—may it sound ever so romantic. I fully understand where this feeling comes from: certainly, there are researchers that act in bad faith and try to publish something they know is wrong. However, my issue with this perspective is that it aligns itself along a low-prevalence event. The common researcher does not have malicious intentions. If they had it would be almost impossible to have any sort of scientific progress! So rather than seeing ourselves as ‘protectors of the realm,’ each one of us should see themselves as the midwife of a paper: our ultimate goal—not necessarily our immediate one—should be to help the research get published eventually. Of course, the paper may not be up to the standards of a certain conference. That’s fair. That happens to all of us. But we should make sure that we are willing to cooperate with the authors1 to ensure that the research sees the light of day at some point in time.
If we write reviews with this mindset, our statements will be imbued with the proper spirit, making it possible to criticise constructively in all situations.
To some extent, this mindset can also be used in other contexts. Code, for example, should also be reviewed with a midwife mindset—even though there may be more reasons to reject pull requests.2 However, even in these contexts, try to be a supporter, instead of a preventer.
In short: do not be a gatekeeper, be a midwife.
-
Of course, cooperation goes both ways, and we cannot protect ourselves against ‘bad faith’ authors. I would argue that these cases are rare. If they are not rare, it might be a sign of a dysfunctional community or system, but handling these cases is beyond the scope of this post. ↩︎
-
For example, feature creep is an issue of code bases that does not necessarily have an equivalent in the scientific community. ↩︎