The contents of the Peer Review Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …
Ensure understanding, reproducibility and replicability
What should you include in your methods section, and how much detail is appropriate?
Why Methods Matter
The methods section was once the most likely part of a paper to be unfairly abbreviated, overly summarized, or even relegated to hard-to-find sections of a publisher’s website. While some journals may responsibly include more detailed elements of methods in supplementary sections, the movement for increased reproducibility and rigor in science has reinstated the importance of the methods section. Methods are now viewed as a key element in establishing the credibility of the research being reported, alongside the open availability of data and results.
A clear methods section impacts editorial evaluation and readers’ understanding, and is also the backbone of transparency and replicability.
For example, the Reproducibility Project: Cancer Biology project set out in 2013 to replicate experiments from 50 high profile cancer papers, but revised their target to 18 papers once they understood how much methodological detail was not contained in the original papers.
What to include in your methods section
What you include in your methods sections depends on what field you are in and what experiments you are performing. However, the general principle in place at the majority of journals is summarized well by the guidelines at PLOS ONE: “The Materials and Methods section should provide enough detail to allow suitably skilled investigators to fully replicate your study.” The emphases here are deliberate: the methods should enable readers to understand your paper, and replicate your study. However, there is no need to go into the level of detail that a lay-person would require—the focus is on the reader who is also trained in your field, with the suitable skills and knowledge to attempt a replication.
A constant principle of rigorous science
A methods section that enables other researchers to understand and replicate your results is a constant principle of rigorous, transparent, and Open Science. Aim to be thorough, even if a particular journal doesn’t require the same level of detail . Reproducibility is all of our responsibility. You cannot create any problems by exceeding a minimum standard of information. If a journal still has word-limits—either for the overall article or specific sections—and requires some methodological details to be in a supplemental section, that is OK as long as the extra details are searchable and findable.
Imagine replicating your own work, years in the future
As part of PLOS’ presentation on Reproducibility and Open Publishing (part of UCSF’s Reproducibility Series) we recommend planning the level of detail in your methods section by imagining you are writing for your future self, replicating your own work. When you consider that you might be at a different institution, with different account logins, applications, resources, and access levels—you can help yourself imagine the level of specificity that you yourself would require to redo the exact experiment. Consider:
- Which details would you need to be reminded of?
- Which cell line, or antibody, or software, or reagent did you use, and does it have a Research Resource ID (RRID) that you can cite?
- Which version of a questionnaire did you use in your survey?
- Exactly which visual stimulus did you show participants, and is it publicly available?
- What participants did you decide to exclude?
- What process did you adjust, during your work?
Tip: Be sure to capture any changes to your protocols
You yourself would want to know about any adjustments, if you ever replicate the work, so you can surmise that anyone else would want to as well. Even if a necessary adjustment you made was not ideal, transparency is the key to ensuring this is not regarded as an issue in the future. It is far better to transparently convey any non-optimal methods, or methodological constraints, than to conceal them, which could result in reproducibility or ethical issues downstream.
Visual aids for methods help when reading the whole paper
Consider whether a visual representation of your methods could be appropriate or aid understanding your process. A visual reference readers can easily return to, like a flow-diagram, decision-tree, or checklist, can help readers to better understand the complete article, not just the methods section.
In addition to describing what you did, it is just as important to assure readers that you also followed all relevant ethical guidelines when conducting your research. While ethical standards and reporting guidelines are often presented in a separate section of a paper, ensure that your methods and protocols actually follow these guidelines. Read more about ethics.
Existing standards, checklists, guidelines, partners
While the level of detail contained in a methods section should be guided by the universal principles of rigorous science outlined above, various disciplines, fields, and projects have worked hard to design and develop consistent standards, guidelines, and tools to help with reporting all types of experiment. Below, you’ll find some of the key initiatives. Ensure you read the submission guidelines for the specific journal you are submitting to, in order to discover any further journal- or field-specific policies to follow, or initiatives/tools to utilize.
Tip: Keep your paper moving forward by providing the proper paperwork up front
Be sure to check the journal guidelines and provide the necessary documents with your manuscript submission. Collecting the necessary documentation can greatly slow the first round of peer review, or cause delays when you submit your revision.
Randomized Controlled Trials – CONSORT
The Consolidated Standards of Reporting Trials (CONSORT) project covers various initiatives intended to prevent the problems of inadequate reporting of randomized controlled trials. The primary initiative is an evidence-based minimum set of recommendations for reporting randomized trials known as the CONSORT Statement.
Systematic Reviews and Meta-Analyses – PRISMA
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is an evidence-based minimum set of items focusing on the reporting of reviews evaluating randomized trials and other types of research.
Research using Animals – ARRIVE
The Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines encourage maximizing the information reported in research using animals thereby minimizing unnecessary studies. (Original study and proposal, and updated guidelines, in PLOS Biology.)
Protocols.io has developed a platform specifically for the sharing and updating of laboratory protocols, which are assigned their own DOI and can be linked from methods sections of papers to enhance reproducibility. Contextualize your protocol and improve discovery with an accompanying Lab Protocol article in PLOS ONE.
Consistent reporting of Materials, Design, and Analysis – the MDAR checklist
A cross-publisher group of editors and experts have developed, tested, and rolled out a checklist to help establish and harmonize reporting standards in the Life Sciences. The checklist, which is available for use by authors to compile their methods, and editors/reviewers to check methods, establishes a minimum set of requirements in transparent reporting and is adaptable to any discipline within the Life Sciences, by covering a breadth of potentially relevant methodological items and considerations. If you are in the Life Sciences and writing up your methods section, try working through the MDAR checklist and see whether it helps you include all relevant details into your methods, and whether it reminded you of anything you might have missed otherwise.
Summary Writing tips
The main challenge you may find when writing your methods is keeping it readable AND covering all the details needed for reproducibility and replicability. While this is difficult, do not compromise on rigorous standards for credibility!
- Keep in mind future replicability, alongside understanding and readability.
- Follow checklists, and field- and journal-specific guidelines.
- Consider a commitment to rigorous and transparent science a personal responsibility, and not just adhering to journal guidelines.
- Establish whether there are persistent identifiers for any research resources you use that can be specifically cited in your methods section.
- Deposit your laboratory protocols in Protocols.io, establishing a permanent link to them. You can update your protocols later if you improve on them, as can future scientists who follow your protocols.
- Consider visual aids like flow-diagrams, lists, to help with reading other sections of the paper.
- Be specific about all decisions made during the experiments that someone reproducing your work would need to know.
- Summarize or abbreviate methods without giving full details in a discoverable supplemental section.
- Presume you will always be able to remember how you performed the experiments, or have access to private or institutional notebooks and resources.
- Attempt to hide constraints or non-optimal decisions you had to make–transparency is the key to ensuring the credibility of your research.