Confirmation Bias
The original error.
1-hour interactive presentation. 3 scaffolding lessons teach how cognitive bias influences research. Participants learn how masking can work to address bias, and some issues that arise when attempting to mask an experiment. Spice up a lab meeting, teach a class, run a skill-building session for your team.
3-hour interactive presentation. 8 scaffolding lessons teach how confirmation bias affects every step of the research process. Participants learn strategies for addressing bias in hypothesis formation, experimental design, and data analysis. Teach one week of class, or run an intensive workshop.
COming soon!
Unit Overview
What you will do
Confirmation bias is the tendency to interpret in a way that aligns with our preexisting beliefs or hypotheses. It distorts how we conduct research and leads to unreliable results. This unit helps you identify confirmation bias and mitigate its effects through practical approaches like masking.
Course highlights
Unit Lessons
(lessons marked with an * are included in the "Meeting" version of the unit)
1. Our biased brains
The Extent of the Problem
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
2. "Favored" vs. alternative hypotheses
Motive & Method
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
3. Researcher degrees of freedom
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
4. Mitigating bias through masking
The Toolbox
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
5. How good is your mask?
Power in Numbers
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
6. Analytical practices to mitigate bias
Power in Numbers
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
7. Data masking in machine learning models
Power in Numbers
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
8. Bonus biases that disrupt research
Power in Numbers
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Solbing Rigor Problems
This unit will explore rigor issues as a cause of unreliable research, and guide you in identifying and seeking out solutions to isolated rigor issues that you may encounter in lab, as well as collective rigor issues that require scientists to work together to solve.
3
Activities
20
minutes
Our biased brains
This lesson introduces the concept of confirmation bias, highlighting how our brains tend to favor information that reinforces existing beliefs. Learn why these biases exist from both psychological and neuroscientific perspectives, and how they can inadvertently affect decision-making in research.
Motive & Method
This lesson focuses on the pitfalls of designing experiments that only test a single, “favored” hypothesis. Learn about and practice developing multiple, mutually exclusive hypotheses to challenge assumptions and reveal hidden biases.
Researcher degrees of freedom
This lesson explains how subjective decisions, ranging from data cleaning to statistical analysis, can unintentionally favor an expected outcome. Learn how “researcher degrees of freedom” can lead to overconfident or skewed results.
Mitigating bias through masking
This lesson introduces the concept of masking (or blinding) in experimental design. Learn how masking can prevent both participants and researchers from inadvertently influencing results with examples from clinical trials.
How good is your mask?
In this lesson, the focus shifts to the evaluation of masking effectiveness. Learn about factors that may inadvertently reveal experimental conditions and how to assess whether masking has been compromised.
How good is your mask?
This lesson introduces differences between exploratory and confirmatory data analysis. Learn how blending these approaches without clear distinction can lead to confirmation bias and discuss strategies to preserve the integrity of research findings.
How good is your mask?
This lesson draws parallels between experimental masking and data leakage in machine learning. Learn about the concept of data leakage and its impact on model evaluation, emphasizing the importance of proper data partitioning.
How good is your mask?
This final lesson broadens the discussion to other biases that can influence research, including conformity bias and cognitive dissonance. Learn how to tie these concepts back to confirmation bias, discovering how multiple biases can compound and affect scientific outcomes.
Unit Overview
Unit Overview
Stop fooling yourself! Diagnosing and Treating Confirmation Bias
1
Activities
1
minutes
Our Biased Brains: Introducing Confirmation Bias
Our Biased Brains: Introducing Confirmation Bias
Tinker with Wason’s 2-4-6 Hypothesis Rule Discovery Task to contend with ambiguous information.
1
Activities
1
minutes