Chapter 2:
Research Methods
Safeguards against Error
Mike Moland, Ph.D.
INTRODUCTORY PSYCHOLOGY
Why do you think Madeline Ennis found an effect of the
homeopathy water when it should be “impossible”?
Why did Benveniste’s team find positive homeopathic
findings under initial testing?
The video indicated that the experimenters knew which
tubes contained the homeopathic water and which
contained the ordinary water. What is the issue here?
When the study was replicated by Randi’s crew using codes
so nobody knew which tubes were which – what happened?
Why do people then find homeopathy useful as a treatment?
Yet homeopathy appears to work even when a placebo
should not. Like when a patient doesn’t even know that
they are taking a medicine. Like with animals. What
might be the explanation here?
David Riley conducted many trials that found positive results
for the homeopathic pollen concentration for hay-fever
sufferers. How does one interpret these findings then?
The video is a fine example of why understanding research methods is vital.
Why do we need research?
Without research designs, even intelligent and
well-educated people can be fooled.
The scientific method is a set of tools that helps us to
avoid being tricked by our own biases.
What biases do you think might have influenced the
results of homeopathy tests in the video seen in class?
We’ll learn what these tools are and how we can use
them to evaluate psychological claims.
But I know it works!
Often our impressions are wrong. Human error.
We need rigorous research methods to
find out whether a technique really works.
Prefrontal lobotomy – What happens
when we rely on our subjective impressions?
Egas Moniz won the Nobel prize (1949) for this
procedure.
Controlled studies done later showed it didn’t work.
Heuristics: Efficient & quick “mental shortcuts”.
Can be systematically & predictably wrong
Availability heuristic – Judging the likelihood of
things by how readily they come to mind.
Are there more words in the dictionary beginning
with the letter R (“rose”, “rock”) or more words with
an R in the third position (“tarp”, “bare,”)? “Hey...this could
be the chief.”
Most participants assert there are more words
beginning with R – but the reverse is true, by at
least two to one (Tversky & Kahneman, 1974).
Representativeness heuristic – judging the likelihood
of an event by its superficial similarity to a prototype.
Daniel Kahneman of Princeton
This may lead one to ignore other relevant information.
University (left) was the first Ph.D.
psychologist to be awarded a
Base Rates: How common a behaviour or Nobel Prize. The Nobel Committee
recognized him for his groundbreaking
characteristic is in the general population. work on the cognitive sources of
human irrationality.
Tversky & Kahneman (1974)
A stranger tells you about a person who is short, slim, and
likes to read poetry, and then asks you to guess whether this person is
more likely to be a professor of classics at a university or a
truck driver. Which would be the better guess?
Most people answer “professor” because the description
seems more representative of scholars than of truck drivers.
Question: How many Canadian universities with classics programs?
Answer: About 45? Question: How many classics profs at each U?
Answer: Um 4? Question: So, 180 classics profs. What % short & slim?
Answer: Let’s say 50%. Question: Of these 90, how many read poetry?
Answer: I’d say half (45 profs). Question: How many truckers are there?
Answer: About 283,000. Question: How many truckers are short & slim?
Answer: Not many – maybe 1 in 8.
Question: Of these roughly 35,000 what % like to read poetry?
Answer: Truck drivers who like poetry? Maybe 1 in 100. O.K. –
I see where this is going – that leaves me with 350 short, slim,
poetry-reading truck drivers.
It takes a wide range of approaches to study the diverse array
of phenomena common to psychology.
Naturalistic Observation - unexpected discoveries!
People are observed in a real-world setting and their
behaviours are carefully recorded (Video? Notes?)
without trying to intervening in it.
Bringing them to the laboratory is unlikely to re-create
the full range of situations that can reveal the behaviour
under investigation. Also, even if we observed participants without their
knowing it, they could still have been influenced by the fact that they were in
a laboratory. Among other things, they may have been nervous or less
spontaneous than in the real world. Cause and effect?
Having more than one observer code the observations
helps assure us that the observations are objective. How?
Another safeguard is to use observers who don’t know the specific
hypothesis that is being tested (they are “blind” to the hypothesis) so that it
is less likely to affect their perception of the events they are observing.
A researcher testing their own hypotheses? Confirmation bias?
Case study designs
Study one person in depth. Generalizable?
Common with rare types of brain damage and other
rare occurring cases (e.g., Phineas Gage).
Can yield existence proofs (that the phenomenon can
happen) but can be misleading and anecdotal.
Correlation studies
Positive correlation – as height increases, so
does weight. In one sample r = +0.47
(Freedman, et all, 1991) for height and weight.
r = +0.23 for positive attitudes toward condom
use and reported use (Herbert et al., 1989).
Negative correlation – as toothbrushing goes Length of marriage correlates with
up, tooth decay goes down. hair loss in men. Does this mean
that marriage causes men to lose
Correlation does not mean causation!!! their hair? Or balding men make
better, more committed husbands?
A Taiwanese study found the best predictor of the
adoption of birth-control use in the home was _____?
Number of electric appliances
Is the solution to teenage pregnancy involve handing out toasters in school?
Answer? Education → high SES → more appliances
An experiment allows for cause and effect conclusions.
Like engineers using a wind-tunnel to simulate varying wind conditions on
wing structures, psychologists investigate by constructing experiments that
simulate important features of our daily lives.
Random selection: Key to generalizability;
ensures every person in a population has an
equal chance of being selected.
Random assignment: All participants have
an equal chance of being assigned to a study condition.
Experimental Group: Receives the manipulation
Control Group: Does not receive the manipulation.
Who invented the control group as we know it?
Psychologist John Coover (1907). “Alternative
explanations of an effect can be ruled out only by
comparing an experimental group with a control group that equals
the experimental group in all respects but the training”.
Independent Variable: Experimenter manipulates (what you do)
Dependent Variable: Experimenter measures to see whether
manipulation had an effect (what you find). It is usually a score.
Placebo effect - improvement resulting
from the mere expectation of improvement.
Subjects must be blind - unaware of
whether they are in the experimental or control group.
Placebos show many of the same characteristics as real drugs.
Nocebo effect - harm resulting from the mere expectation
of harm (e.g., voodoo doll phenomenon)
Experimenter expectancy effect - phenomenon in
which researchers’ hypotheses lead
them to unintentionally bias a study.
Clever Hans, the mathematical horse.
Double-blind design - neither
researchers nor subjects know who is
in the experimental or control group
addresses the experimenter expectancy effect.
Self-report measures - questionnaires
assessing a variety of characteristics (e.g., interests, traits)
Surveys - measure opinions, attitudes
Reliability: consistency or repeatability of the measurement.
Validity: Does it measure what it’s supposed to measure?
Easy to administer, direct assessment
of person’s state
Cons? (Do you masturbate?)
Potential for dishonesty
Response sets - tendencies of research subjects to distort
responses. Positive impression management? Malingering?
Why did you do that? Verbal reports & behaviour often don’t
match.
Unrepresentative samples – mail-in, call-in
Response options – Limit responses
Anchoring effects – Mississippi (>500 miles)? (<3000 miles)? Response Options
Survey tapped information about ones eccentricities
They found that __% roll toilet paper over the spool
68%
That __% squeeze toothpaste from the top.
79%
__% look behind shower curtain when in
someone else’s bathroom.
7%
__ % like the way they look nude.
41%
__% eat corn-on-the-cob in circles (not side to side).
80%
Sent surveys to 25,000 people, with 7000 returned
How representative is this?
Are the 1 in 700 readers who participated representative of the
699 in 700 who did not?
Descriptive statistics: helps to describe or
summarize data in a meaningful way.
Central tendency: where the group tends to cluster
Mean - average of all scores.
Median - middle score in the data set.
Mode - most frequent score in the data set.
Range - difference between highest and lowest scores.
Standard deviation - measure of variability that takes
into account how far each data point is from the mean.
Inferential statistics: allow us to determine how much
we can generalize findings from our sample to the full
population. If we find a difference between groups (e.g., a
female group shows higher scores on extroversion than a
male group in our research). How can we rule out the
This is data above is
possibility that a small difference in our sample is due to chance? from homeopathic
research seen in the
Statistical significance: To be statistically significant, the homeopathic video
watch in class. Does
finding would have occurred by chance less than 1 in 20 times (p<.05). this figure indicated
statistical
Practical significance: Real-world importance. significance?
1. What is one take-home message from the discussion of
autism and facilitated communication?
a. The scientific method is not an effective means for finding solutions
for persons who live with autism and other psychological disorders.
b. Even in the face of overwhelming evidence, some people won’t abandon
their erroneous beliefs.
c. Autistic children want to communicate with their parents but need someone
to facilitate the process.
d. Psychological research is dangerous because it allows anyone to find
support for any idea or opinion.
2. Sahar attended a workshop on how crystals have healing power, can
eliminate “blood sludge,” and cure mental illness. The workshop facilitator
presented clinical observations of several patients to demonstrate the
effectiveness of crystals. What should Sahar keep in mind here?
a. In the absence of systematic research, clinical observations are not
sufficient evidence for their effectiveness.
b. Clinical observations are a powerful source of evidence and demonstrate
treatment effectiveness.
c. Patients’ own reports that crystals cured their cancer or depression provide
empirical evidence of their effectiveness.
d. The workshop facilitator appears credible, so his claims must be true.
3. A group of student researchers divide up the different times and buildings on
their campus to attempt to determine when people will hold a door open for
another person. These student researchers are most likely to use which
research method design when conducting their study?
a. Correlational design b. Experimental design
c. Case study design d. Naturalistic observation design
4. Dr. Didus has diagnosed a patient with dissociative identity disorder (DID), a
very rare type of dissociative disorder. He observes the behaviour of his
patient and her alter personalities and discovers that every time he asks her
about sexual experiences, an alter that is a male takes over as the dominant
personality. Based on his case study, what can Dr. Didus conclude about DID
in general?
a. Patients with DID cannot cope with discussing uncomfortable experiences.
b. Sexual experiences are linked with the presence of alter experiences.
c. Opposite gender personalities take over to protect the host personality when
talking about sex.
d. No conclusions can be drawn about DID without systematic research on this
population.
5. A key disadvantage to self-report measures is that:
a. they are less effective than experiments in accurately
predicting peoples’ behaviour.
b. respondents are not always honest in their answers.
c. observing behaviour leads to changes in behaviour.
d. demand characteristics can bias participants’ answers.
6. If you are interested in examining the relationship between the number of
class days missed and one’s subsequent semester grade point average, you
would be best served to use a(n) __________ to study this question.
a. experimental design b. naturalistic observation design
c. case study design d. correlational design
7. Which of the following correlations represents the weakest degree of
relation between two variables?
a. Daily calcium intake and bone mass density, r = +.11
b. Number of cigarettes smoked per day and incidence of lung cancer, r = +.39
c. Degree of exposure to lead and IQ scores in children, r = –.12
d. Hours of exposure to media violence and aggressive behaviour, r = +.31
8. You read a research study that claims to have found
a correlation of r = –.43 between smoking and vegetable
consumption. This means that:
a. the less you smoke, the fewer vegetables you eat.
b. the more you smoke, the fewer vegetables you eat.
c. the more you smoke, the more vegetables you eat.
d. smoking and vegetable consumption are unrelated.
9. Suppose a researcher wants to see if those students who highlight their
textbook as they read will perform better on the psychology mid-term
compared to those students who do not highlight. What is the dependent
variable?
a. Psychology mid-term score b. Highlighting of text
c. No highlighting of text d. Grade-point average
10. A British literature instructor examines the number of class periods his
students have missed by mid-terms and has the following data: 1, 0, 10, 0, 2,
1, 0, 0, 5, 2, 3, 0, 0, 0, 1, 1, 2, 3, 1. What is the mode for this data set?
a. 2.5 b. 0 c. 10 d. 1
