Name: _________________
AP Statistics Handout: Lesson 4.4
Topics: blocking, matched pairs, placebo, blinding, generalizing study results
Lesson 4.4 Guided Notes
Blocking
The Jenn/John Study
1
In this study, experimenters printed up copies of fake application
materials for a science lab manager job. All applications were
identical, except for one thing: the name. On half the applications,
the applicant’s name was listed as “John.” The others had “Jennifer.”
Experimenters found science lab faculty members and randomly sent
them either a “John” or “Jennifer” application. The faculty members independently rated the applicant’s
“hireability” (1-7 scale). Then, experimenters compared “hireability” ratings across the two groups.
Imagine the experimenters got 8 faculty to evaluate these
application materials (4 for “John” and 4 for “Jennifer”). The
hireability ratings from the faculty are displayed to the left.
1) Based on these results, would you say that there’s
substantial evidence of hiring bias? Why or why not?
Imagine the faculty were from both Tier 1 and Tier 2 Universities. Generally, Tier 1 Universities are more
prestigious and competitive, so it's more difficult to get a lab manager job at a Tier 1 school.
At left are the same results, but now broken down by
University Tier.
Tier 1 -
Tier 2 -
2) Displayed this way, do you see more evidence of hiring
bias? Why or why not?
1Example inspired by Advanced High School Statistics. Original study: Moss-Racusin, C., Dovidio, J., et al. “Science faculty’s subtle gender biases favor male
students.” PNAS October 9, 2012 109 (41) 16474-16479; https://doi.org/10.1073/pnas.1211286109
Material adapted from the Skew The Script curriculum (skewthescript.org)
2
To compare within school Tier, we need to implement a…
Randomized Complete Block Design: experimental units are first blocked (_________________) by a
similar trait that may affect response. Then, units from each block are randomly assigned to treatment.
• This is the _______________________________ of experiments.
• It reduces variation between treatment groups at the start of the experiment. This makes it
easier to show that differences in response are really _______________________________,
rather than chance variation in the random assignment.
3) Describe how you would implement a randomized block design for the Jenn/John study. Imagine
there are 12 faculty (6 from Tier 1 schools, 6 from Tier 2 schools) and that you block on University Tier.
Matched Pairs Design
Matched Pairs Design: a type of randomized blocked experiment in which each block is composed of
_______ similar experimental units (a “matched pair”).
• Often, the “matched pair” is simply the same experimental unit receiving both treatments. The
____________ of the treatments is randomized.
The 2015 Journal of the American Medical Association Depression Study
2
In this study, researchers wanted to test if taking a “fake pill” would actually alleviate depression
symptoms, even though the pill had no active ingredients. 35 people enrolled in the study. All had major
depression and none were taking any medications. Because depression varies greatly between
individuals, the researchers implemented a matched pairs design.
2 Simplified version of: Association Between Placebo-Activated Neural Systems and Antidepressant Responses: Neurochemistry of Placebo Effects in Major
Depression. Peciña M, Bohnert AS, Sikora M, Avery ET, Langenecker SA, Mickey BJ, Zubieta JK. JAMA Psychiatry. 2015 Sep 30:1-8. doi:
10.1001/jamapsychiatry.2015.1335. [Epub ahead of print]. PMID: 26421634.
Material adapted from the Skew The Script curriculum (skewthescript.org)
3
4) Describe how you would implement a matched pairs design for the depression study.
Depression study results
1. On average, participants reported less severe depressive symptoms after their week with
the fake pill.
2. On average, participants showed “increased µ-opioid receptor brain activity in regions of the
brain associated with emotion and stress regulation.”
o Belief in the pill caused a ____________________________
“Fake pills” have also have also shown3 significant beneficial effects for….
• Migraines
• Blood Pressure
• Asthma
• Arthritis
• Many other ‘physical’ illnesses
Blinding and Placebo
Placebo: An ________________ treatment (e.g. sugar pill or salt water IV drip)
Placebo effect: when subjects’ belief of receiving an active treatment leads to a measured response,
even though the treatment is ___________________________.
Single-blind study: _______________ the subjects or the researchers are unaware of who receives
active treatment or placebo.
Double-blind study: ________ the subjects and the researchers are ___________________ of who
receives active treatment or placebo.
5) Why is it beneficial to blind subjects? Why is it beneficial to blind researchers?
3 Source: https://www.ncbi.nlm.nih.gov/books/NBK513296/
Material adapted from the Skew The Script curriculum (skewthescript.org)
4
Lesson 4.4 Discussion
Summary of the Jenn/John study
results:
John Jennifer Difference
Hireability 3.78 2.93 -0.85
Salary $30,238 $26,508 -$3,730
Estimates based on publicly
available data from the study.
Generalization: using study results to make inferences about a _________________________.
Discussion Question: In the Jenn/John study, all faculty participants were from one of three science
departments: biology, chemistry, or physics. Can we generalize the result of gender hiring discrimination
to all scientific subjects? Explain your reasoning.
Lesson 4.4 Post-Discussion Notes
Were subjects randomly assigned to treatment?
Subjects were randomly assigned to Subjects were not randomly assigned to
randomly sampled?
groups (experiment) groups (observational study)
Were subjects
Subjects were randomly Generalize to the population: ___ Generalize to the population: ___
sampled from pop. Determine cause and effect: ___ Determine cause and effect: ___
Subjects not randomly Generalize to the population: ___ Generalize to the population: ___
sampled from pop. Determine cause and effect: ___ Determine cause and effect: ___
Lesson 4.4 Practice
Teachers: We recommend providing additional practice exercises from your AP Stats textbook or from
prior AP Stats exams. The following textbook sections and AP exam questions are aligned to the content
covered in this lesson.
• The Practice of Statistics (AP Edition), 4th-6th editions: sections 4.2 – 4.3
• Stats: Modeling the World (AP Edition), 4th/5th editions: ch 12, 3rd edition: ch 13
• Statistics: Learning from Data (AP Edition), 2nd edition: sections 1.4 – 1.5
• Advanced High School Statistics, section 1.5
• AP Exam Free Response Questions (FRQs): 2007 Form B Q3, 2007 Q2, 2019 Q6 (part a)
Material adapted from the Skew The Script curriculum (skewthescript.org)
