Target Topics

This is a work-in-progress, CENTER-based vision of topics that we intend to cover. It’s based on extensive interviews with practitioners across many fields of science.

The origin of hypotheses

  • Performing comprehensive literature reviews
  • Discerning differences between hypothesis-generating (exploratory) and hypothesis-testing (confirmatory) research
  • Formulating testable hypotheses to clarify questions
  • Using mathematical models to make experimental predictions
  • The types of origins of hypotheses
  • Confounding as failure mode for causal inference (do we need to randomize)
  • Creativity

Experiments to test Hypotheses

  • Pre-registering experimental protocols, plans, and analyses
  • Biases induced by inclusion and exclusion criteria
  • Calculating necessary sample size and the mistaken inferences that come with not doing so
  • Proper Randomizing, including Blinding investigators during experimentation and analysis
  • Identifying assumptions and sources of error
  • Bad model systems

Analyzing data to test the idea

  • Creating an analysis plan / relating data to hypotheses
  • Handling outliers and resulting biases
  • Knowing when to use frequentist vs Bayesian statistics
  • Understanding null hypotheses, p-values, and statistical significance
  • P-hacking / Experimenter degrees of freedom
  • P-hacking/ overfitting in ML
  • HARKing (hypothesizing after results are known) / Garden of Forking Paths
  • Model comparison

Report it

  • Reporting project workflows, methods, and divergence from plans
  • Displaying data transparently in figures and presentations
  • Sharing data well /standardized variables
  • Sharing code well
  • Identifying all relevant experimental details

Creating good processes

  • Creating standard operating procedures for laboratory workflows
  • Using emerging technologies to improve rigor and transparency
  • Using validated resources, methods, and outcome measures
  • Reproducing and replicating experiments /Reducing publication bias
  • Open science
  • Fighting bad rigor
  • Keeping proper logs/ notebooks
  • Validating resources