Peer review: what is it?

Peer review (sometimes also called ‘expert review’ or ‘merit review) implies that colleagues (‘peers’) assess the research of other scholars, groups of scholars or entire institutions. Peer review is considered an effective way of safeguarding the quality of research. All researchers are involved in peer review: as the person to whom a review applies and (once a researcher has acquired some research experience) also as a reviewer.

Peer review has a central place in decision-making processes that are crucial for the research and the researchers. It is used – whether or not in combination with other review methods – among other things to determine:

which manuscripts (journal articles, monographs,...) will be published
which contributions and posters are presented at a conference
which grant and project applications, applications for mobility, etc. are funded
which research groups and institutions are funded or accredited
who will be recruited or promoted
to whom a scientific prize or distinction is awarded

Quality and integrity

Peer review should be carried out with the greatest possible care and should meet the following criteria with regard to quality and integrity:

Expertise: The reviewers possess sufficient knowledge to carry out the review. Depending on the nature of the review, their expertise goes beyond scientific knowledge and includes for example the knowledge and experience needed to assess various forms of impact. The reviewers receive clear instructions (and possibly training) on the criteria they should pay attention to during the review, their responsibilities and the expectations of the organisers of the peer review (timelines to be respected, preparation of a report, etc.).
Diversity: The selection of reviewers calls upon the various talents present in the research community. Researchers are not a priori excluded from review work on the basis of gender, ethnicity, etc.
Confidentiality: The contents of the manuscripts, proposals and applications to be reviewed are treated confidentially by the reviewers and all other actors involved in the peer review process.
Transparency: The reviewers are selected according to clear and known criteria. Proposals, manuscripts, etc. are reviewed according to clear and known criteria. Without prejudice to confidentiality, applicants, authors, applicants etc. receive feedback on the peer review process or the wider evaluation procedure of which the review is part. Procedures must also be in place to allow researchers to respond to or appeal against a (decision based on) peer review.
Impartiality: All dossiers are treated equally and fairly. The reviewers act in full independence. They report possible conflicts of interest and, if necessary, withdraw as a reviewer. Reviewers do not allow themselves to be influenced by conscious or unconscious bias and do not discriminate (e.g. by age, gender, discipline). Where possible, mechanisms are built in to avoid bias. The review of manuscripts, for example, is often done single- or double-blind (in the first case, the authors do not know the identity of the reviewers; in the second case, the reviewers do not know the identity of the author(s) either).
Appropriateness: The peer review process is adapted to the context in which the evaluation is conducted (e.g. appropriate to the specific project call).
Efficiency: The peer review process is conducted in an efficient and as simple as possible manner. Procedures, expectations and responsibilities are defined in advance and made known to all actors involved in the peer review process. The process is administratively supported and monitored. Sufficient people and resources are provided for this purpose. The reviewers commit themselves to carry out a review within an agreed timeframe. Especially for manuscripts, a lengthy peer review procedure should not lead to delays in the publication process.

Limitations

Peer review that meets the above criteria is generally accepted by the research community as an effective way of detecting quality research (in the sense that it is used in a specific review). Nevertheless, peer review also has a number of limitations, including:

Researchers spend a lot of time on reviews, without getting recognition and appreciation – let alone financial compensation. This leads to a certain 'reviewer fatigue', whereby researchers no longer accept review work, no longer succeed in submitting reviews on time, etc.
The same reviewers – often established in research – are often called upon, which makes it difficult for young researchers to gain experience as reviewers.
Bias cannot be excluded. Reviewers also appear to be rather conservative in their assessments. Interdisciplinary research, highly innovative and experimental research and risky research therefore have less chance of being funded and published.
It is not always easy to find enough suitable experts, especially in small disciplines. There is often a (potential) conflict of interest, because researchers work in the same group, are co-authors of publications, etc. In scientific fields with a small number of researchers, it is hard to guarantee the anonymity of an author or reviewer, because the reviewers know the authors and vice versa.
Peer review is not aimed at detecting breaches of research integrity (although opinions are divided on this). Due to the large number of articles that are withdrawn after publication because of breaches of research integrity, mechanisms are increasingly being put in place to increase the chances of detecting fraud or breaches. Thus, a manuscript is often screened for 'image manipulation' before peer review, the raw data are also requested upon submission, etc.
In the evaluation of project proposals, peer review is suitable for identifying substandard and excellent proposals. However, it is not an efficient way to determine which proposals will be funded in the zone in between – the project proposals of equal (or similar) quality. Reviewers of project proposals get frustrated when the available budget is not sufficient to award funding to all eligible proposals. Experiments are therefore being made with alternative ways of determining which projects of equal quality receive funding, e.g., partial randomisation.
In peer review for publications, the large number of manuscripts to be reviewed slows down the system. As a result, it can take a long time for a publication to appear, delaying the dissemination of new knowledge. To make it possible to publish research results quickly, some online publishing platforms opt for post-publication peer review.