Substantive analysis

Before peer reviewing, please take a look at the review protocol used by SaintScience across its publications

Theory and factual information

All article formats assume theoretical background knowledge, whether explicitly or implicitly. As a reviewer, it is key to assess the truth of all premises made in the article. Premises are statements made that assert something in support of a conclusion in an argument. In order to draw true or defeasible conclusions from valid arguments, its premises must hold true.

Consequently, in checking premise truth, one could encounter three scenarios:

The premise is factually incorrect and simply false, i.e. Mars has a larger diameter than the Earth’s. In this instance, the statement would be flagged in comments. The reviewer should closely read the remainder of the argument to check whether the argument is resting upon the false statement. Any large passages that contain multiple serious factual errors should be flagged.
The author makes an erroneous assumption. Often when writing and speaking, humans omit underlying premises in an argument because they are assumed to be self-evident or common-sense. However, when reviewing an article, the reviewer must read between the lines and try to discern whether the author is making any assumptions that are factually false. For instance, the argument I cannot make dinner tonight since I do not have any tomatoesisinvalid as it stands without the hidden assumption,I can only make dinner if I have tomatoes.The argument is then valid but unsound, for evidently tomatoes are not essential for making dinner. While the argument given is clearly false, sentences obscured in complex syntax and jargon may be harder to check for false assumptions.Tackling erroneous assumptions In this instance, we recommend taking a step back from the sentence/paragraph and noting down on paper the salient points the author is attempting to make, including any underlying assumptions needed to support these points. While making sure to consult existing literature to clarify what is factually correct, flag any points in the article that rest on false assumptionS.
The statement is ambiguous or vague. If it is impossible to tweeze apart the meaning of a statement, this should be flagged. This could include•Use of object pronouns or demonstratives like it/they and this/these without clear reference to the reference subject•Generalisations and brush-stroke comments. Be aware of the compounding fallacy. While a member of a larger class may possess a certain property, it does not entail that all members have the same property. For instance, if one were to say mammals give birth to live young, this is factually ambiguous, (eutherians like primates do but metatherians like platypus do not). Did the author mean ‘some mammals’, ‘most mammals’ or ‘all mammals’?

Therefore be sceptical of statements lacking modifiers like ‘all’, ‘every’, ‘most’ or ‘some’ when the accuracy of the statement depends on having one. This should be flagged.2.2. Technical vocabulary usage Use of precise vocabulary that is context-specific and relevant to the audience the article is addressing is key. As a peer reviewer, one must ensure that (1) technical vocabulary is used accurately and (2) level of jargon is appropriate to the audience background.

Comment and Opinion pieces and Research abstracts

Jargon should be explained in sufficient detail for an individual with broad scientific training to understand when first used. If the theory behind the vocabulary usage is too extensive to summarise, a clear mention to external resources should be made in the article with relevant reference(s). Check for accessibility and flag any unnecessary jargon for broad understanding or demand clearer explanation

Minireviews, Reviews and Primary Research

The article is aimed at a more specialised audience. Consequently, more extensive use of assumed jargon is acceptable. If you, a member with similar knowledge to the article, struggle to read the article due to dense verbiage, this should also be flagged as a recommendation for the author to clarify their thought process.

Figures and results

Figures are intended to draw attention to relevant points in an article, present results or illustrate a concept more clearly. Reviewers should

Ensure that all figures are accurate. As described above, checking for any technical errors/inaccuracies is key.
Ensure figures are properly cited and can be relocated if adapted from another source
Figure clarity. Is the figure clear? How crowded or text-heavy is it?
Accuracy of the figure legend. Often, legends fail to describe key features of the figure panels themselves or omit important information to understand them. The figure legend should be read to ensure its contents match the figure with sufficient detail that it can be read independently of the main text and still be understood. Likewise, if the reviewer feels that a figure should be inserted to explain a particular concept, this should also be mentioned.

Argument validity and conclusions drawn

A peer reviewer should check not only the accuracy and truth of the statements made in an article, but its logical structure. Peer reviewers should check that

Conclusions follow logically from the statements made in the article. Does the conclusion, i.e., in the discussion section reach beyond its premises? If the conclusion makes a generalisation from results or data that appears unfounded, this should be flagged. Reviewers should check existing literature and citations made to assess the feasibility of conclusions made
The author has not committed any fallacies or clear errors in reasoning. This resource, https://www.logical-fallacy.com/articles/list-of-formal-fallacies/can help identify the most common errors in reasoning you might encounter. These should again be flagged.

Methods and results

While more applicable to primary research papers and meta-analyses, peer reviewers must assess the quality of the methodology of a paper, namely, if the methods described are used, are they necessary and jointly sufficient to answering the questions laid out in the introduction? If the methodology is fundamentally flawed and cannot answer these questions or shows irrelevance, this must be flagged immediately and sent back to the school managing editor.

Some things to check:

Experimental design – have authors/experimentalists tried to control for confounding variables? Are suitable controls in place?
Sample size –risk of sampling error, pseudoreplication or lack of technical repeats?
Output variable(s) –is the measuring apparatus or system of measurement appropriate and suitable for answering the question(s) laid out in the introduction? Read the method closely for technical specifications, statistical analyses completed, animal/cell systems if relevant using your background in the subject and existing literature.
Sufficient detail in methodology. Given the information presented, could another experimenter reproduce the experiment perfectly? For meta-analyses, this would include details on search parameters, indexing platforms (Web of Science, PubMed, etc.)
Result presentation. Using the methods given, can the results given be attained? All results and figures presented should have been generated using the methods given.

Citations

Citations are what enable researchers to see further than others by “standing on the shoulders of giants” (Sir Isaac Newton in conversation with Robert Hooke). Citations are allusions made to literature supporting an assertion made in the article. As a peer reviewer, one must ensure that

The citation style is correct and free from errors. SaintScience uses IEEE format (please visit the Editorial house-style for further details). Therefore, the first feature to check for is consistency in the referencing style.
Citations are made almost exclusively to primary literature and secondary literature to a lesser extent, depending on the article format. Unlike media reports, our articles should synthesise the findings from the source rather than using filtered and reported articles. Primary sources include original research papers and meta-analyses, whereas secondary sources comprise reviews and comment articles. Tertiary pieces like news reports or bulletin updates should not be used as principal evidence for an assertion made in the article.
All assertions made in the article that go beyond common-sense or background knowledge should have citations. While reading the article, peer reviewers should check whether any citations appear to be lacking in the article. if the peer reviewer discovers a statement made that does not rest on commonsense or background knowledge appropriate to the readership, this should be flagged. The reviewer should also search for relevant literature to recommend a supporting citation to the author.