This is a work-in-progress, CENTER-based vision of topics that we intend to cover. It’s based on extensive interviews with practitioners across many fields of science.
The origin of hypotheses
- Performing comprehensive literature reviews
- Discerning differences between hypothesis-generating (exploratory) and hypothesis-testing (confirmatory) research
- Formulating testable hypotheses to clarify questions
- Using mathematical models to make experimental predictions
- The types of origins of hypotheses
- Confounding as failure mode for causal inference (do we need to randomize)
- Creativity
Experiments to test Hypotheses
- Pre-registering experimental protocols, plans, and analyses
- Biases induced by inclusion and exclusion criteria
- Calculating necessary sample size and the mistaken inferences that come with not doing so
- Proper Randomizing, including Blinding investigators during experimentation and analysis
- Identifying assumptions and sources of error
- Bad model systems
Analyzing data to test the idea
- Creating an analysis plan / relating data to hypotheses
- Handling outliers and resulting biases
- Knowing when to use frequentist vs Bayesian statistics
- Understanding null hypotheses, p-values, and statistical significance
- P-hacking / Experimenter degrees of freedom
- P-hacking/ overfitting in ML
- HARKing (hypothesizing after results are known) / Garden of Forking Paths
- Model comparison
Report it
- Reporting project workflows, methods, and divergence from plans
- Displaying data transparently in figures and presentations
- Sharing data well /standardized variables
- Sharing code well
- Identifying all relevant experimental details
Creating good processes
- Creating standard operating procedures for laboratory workflows
- Using emerging technologies to improve rigor and transparency
- Using validated resources, methods, and outcome measures
- Reproducing and replicating experiments /Reducing publication bias
- Open science
- Fighting bad rigor
- Keeping proper logs/ notebooks
- Validating resources