Writing Methods for Reproducibility

Methods is where we spend the most of our experimental time, but the least of our writing time. Well-written methods are necessary to replicate findings, assess the rigor of an experimental design, and determine the breadth of data interpretation. Detailed methods allow subsequent research to quickly build on your findings and to interpret whether contradictory results are due to methodological differences. Moreover, publishing protocols is a critical component of open science, as the methods are themselves data. Here, I’ll describe a protocol for writing reproducible methods.

  1. Design the experiment. Before beginning the experiment, outline the complete experimental design. Write down all necessary materials, equipment, organisms, and procedures.
    1. Use experimental design tools to map out all manipulations, measurements, and comparisons.
    2. Determine what data will be collected and how it will be stored.
    3. Pre-register your study, either publicly to commit to separating exploratory from confirmatory experiments or privately to plan your analyses. Several journals have committed to reviewing pre-registered studies and publishing the resulting article regardless of the outcome.
    4. Consider each design choice as a piece of data. How will it affect the interpretation of your results?
  1. Write as you go. Write methods while you do experiments so they are fresh in your mind. This will give you a headstart on the manuscript and prevent the methods from being relegated to the final pre-submission writing crunch.
    1. Use protocols as a teaching tool. Write protocols with sufficient detail so that a new trainee in your lab could understand it. Ask trainees learning a procedure to read the existing protocol in advance, then expand with their own notes to consolidate their learning.
    2. Make common protocols in your lab into living documents which are expanded upon by each subsequent trainee.
    3. Use tools like protocols.io to store protocol changes using version control prior to data collection. Track which data were generated using which protocol version.
  1. Include enough detail so a field expert could implement your methods. Consider the methods you’ve learned through hands-on training. How can scientists who don’t have a skilled collaborator to train them learn that technique? What methods from other papers do you wish you could implement, if you had sufficient detail? What methods are derived independently in each lab, effectively re-inventing the wheel?
  1. Use checklists to include all necessary details. Checklists can serve as outlines for methods, speeding up writing. They can also disclose methods components that may seem standard, but can affect results interpretation, such as data preprocessing steps, sex of animal subjects, or circadian timing of data collection.
    1. Use checklists generated by consortia for life sciences, biological research, or health research.
    2. Use checklists specific to journals, such as eLife, Cell Press, and Nature Portfolio.
    3. Make critical materials findable by including Research Resource Identifiers (RRIDs) and depositing generated resources, like sequences, into databases.
  1. Share protocols independently from papers. Deposit protocols on repositories such as protocols.io or Protocol Exchange. This generates a DOI, which can be cited in methods sections of subsequent papers.
    1. Use protocol templates to convey critical information not often found in methods sections, like photographs, diagrams, and safety warnings.
    2. Include troubleshooting steps to guide a novice through the procedure. What technical tips have you learned that would have saved you time had you known them in advance?
    3. Include your decision making process and alternatives. Is it critical that this particular reagent be used? Have you tried others, and if so, what was the result?
    4. Original methods can be peer reviewed and published in journals like JOVE, Bio-protocol, and PLOS ONE and shared on open science databases like Open Neuroscience.
  1. Link your protocols, code, and data. This enables you to combine the methods, the data collected using those methods, and the code used to generate publication figures into one reproducible package.
  1. Iterate for the next paper. Build on previous protocols.

Learn more about writing reproducible methods from guides at PLOS and Nature.

Emily A. Aery Jones

Emily Jones is a postdoc in Dr. Lisa Giocomo’s lab at Stanford University studying how medial entorhinal cortex might act as an adaptable substrate to support spatial learning over days. Specifically, she is interested in how organization of spatial information into sequences during task engagement and replay of these sequences during task disengagement might change as learning progresses and might be required for learning.

Website: https://emilyjon.es
Twitter: @emilyaeryjones

Leave a Reply

Your email address will not be published. Required fields are marked *