Research Method

Data Collection
Topics
• Data Collection Methods
• Interviews
• Observation
• Questionnaires
• Experiments
Data Collection
• Interviews
• Observation
• Questionnaires
• Experiments

3
Choice of method depend on:
• Objective(s) of the study
• Research questions
• Research strategy
• Facilities available
• Degree of accuracy required
• Type of data required
• Time span of the study
• Expertise of the reviewer
• Other costs and resources associated with and available for
data gathering
Interview
• Interview: a guided, purposeful conversation between two or more people.
• Individual or group interviews
• Unstructured or structured
• Conducted face to face, by telephone, or online
• The unstructured interview is to explore and probe into the several factors in
the situation that might be central to the broad problem area. This could result
in the identification of several critical factors in the situation or the variables that
needed greater focus.
• The structured interviews elicit more in‐depth information on the variables (to
identify the critical problem and ways of solving it).
• In applied research, a tentative theory of the factors contributing to the problem
is often conceptualized on the basis of the information obtained from
unstructured and structured interviews.
Interview Content
• The content can be prepared in advance:
• an introduction: the interviewer introduces him‐ or herself, the purpose of the interview,
assures confidentiality, asks permission to record the interview;
• a set of topics (usually questions) in a logical order: first “warm‐up” questions (which are
easy to answer and non‐threatening) and then the main questions covering the purpose of
the interview;
• suggestions for probing questions: follow‐up questions that are used when the first answer is
unclear or incomplete, the interviewer does not fully understand the answer, or in any other
case where the interviewer requires more specific or in‐depth information.
• Probing tactics:
Silence Repeating the answer
I ’ m not quite sure I understood . . . Could you . . . So what I hear you saying is .
..
Could you please tell me more about . . . Could you give an example?
Could you go over that again? Anything else?
Bias
• Bias: errors or inaccuracies in the data collected.
• Bias could be introduced by the interviewer, the interviewee, or the situation.
• nonparticipants
• trust levels and rapport established (do your homework!)
• the physical setting of the interview
• Strategies to reduce bias:
• Establishing credibility & rapport, and motivating individuals to respond
• The questioning technique:
• Funneling (broad to narrow)
• Unbiased questions
• Clarifying issues
• Helping the respondent to think through issues
• Taking notes
Focus Groups and Expert Panel
• “Focus group research” is a generic term for any research that studies how groups of people
talk about a clearly defined issue. Focus groups consist typically of eight to ten members
with a moderator leading the discussions on a particular topic, concept, or product.
Members are generally chosen on the basis of their familiarity with the topic on which
information is sought.
• The focus sessions are aimed at obtaining respondents’ impressions, interpretations, and
opinions, as the members talk about the event, concept, product, or service. The moderator
plays a vital role in steering the discussions in a manner that draws out the information
sought, and keeps the members on track.
• An expert panel: a group of people specifically convened by the researcher to elicit expert
knowledge and opinion about a certain issue. The criteria for qualification as an expert are
many and varied, but the expert panel usually comprises independent specialists, recognized
in at least one of the fields addressed during the panel sessions. Expert panels may thus bring
together a wide variety of experts, including scientists, policy makers, and community
stakeholders.
Observation
• Observational methods are best suited for research requiring non‐self‐report
descriptive data; that is, when behavior is to be examined without directly
asking the respondents themselves.
• Observational data are rich and uncontaminated by self‐report bias.
• Observation concerns the planned watching, recording, analysis, and
interpretation of behavior, actions, or events:
• control (are the observations conducted in an artificial or in a natural setting?)
• whether the observer is a member of the group that is observed or not (participant
versus nonparticipant observation)
• structure (to what extent the observation is focused, predetermined, systematic,
and quantitative in nature)
• concealment of observation (are the members of the social group under study told
that they are being studied or not?)
Observation
Advantages Disadvantages

• Directness • Reactivity (the extent to which the

• Possible to observe certain groups of observer affects the situation under
individuals: study)
• very young children • Bias
• extremely busy executives • Time consuming
(from whom it may be otherwise
difficult to obtain information)
Structured Observation
• Structured observation is largely quantitative in nature.
• The specific instrument for collecting the necessary data is your
coding scheme. Coding scheme should be valid and reliable.
• Validity: the extent to which observations accurately record the behavior in
which researchers are interested.
• Reliability: the consistency of observations, usually whether two (or more)
observers, or the same observer on separate occasions, observing the same
event attain the same results.
Field Notes
Good field notes:
• use exact quotes when possible;
• use pseudonyms to protect confidentiality;
• describe activities in the order in which they occur;
• provide descriptions without inferring meaning;
• include relevant background information to situate the event;
• separate one’s own thoughts and assumptions from what one actually observes;
• record the date, time, place, and name of researcher on each set of notes.
Questionnaires
A questionnaire is a preformulated
written set of questions to which
respondents record their answers,
usually within rather closely
defined alternatives.

Steps:
1. Determine the content of the
questionnaire
2. Determine the form of
response
3. Determine the wording of the
questions
4. Determine the question
sequence
5. Write cover letter
Questionnaire Content
• Framework
Need information for all constructs in framework

• Measurement: Operationalizing
• Objective construct:
• 1 element/items
=> 1 question
• Subjective construct:
• multiple elements/items
=> multiple questions
2. Response Format
• Closed vs. Open-ended questions
• Closed questions
• Helps respondents to make quick decisions
• Helps researchers to code
• Open-ended question
• First: unbiased point of view
• Final: additional insights
• Complementary to closed question: for interpretation purpose
3. Question wording
• Avoid double-barreled questions
(combined the two questions, would confuse the respondents and
obtain ambiguous responses)
• Avoid ambiguous questions and words
• Use of ordinary words
• Avoid leading or biasing (loaded) questions
• Social desirability
• Avoid recall depended questions
More on Question Wording
❑Use positive and negative statements
• Dresdner delivers high quality banking service
Dresdner has poor customer operational support
• Avoid double negatives

❑Limit the length of the questions

Rules of thumb:
• < 20 words
• < one full line in print
4. Question Sequence

Personal and sensitive data at the end

5. Cover Letter
• The cover letter is the introductory page of the questionnaire

• It includes:
• Identification of the researcher
• Motivation for respondents to fill it in
• Confidentiality
• Thanking of the respondent
Cross-cultural Research
• Important issues for cross‐cultural data collection:
• response equivalence
• timing of data collection
• the status of the individual collecting the data
• Different languages are spoken in different countries, it is important to ensure
that the translation of the instrument to the local language matches accurately to
the original language.
• Pepsi’s “Come alive with the Pepsi generation” when translated into Chinese means “Pepsi
brings your ancestors from the grave.”
• Frank Perdue’s chicken slogan “It takes a strong man to make a tender chicken” translates in
Spanish to “It takes an aroused man to make a chicken affectionate.”
• When American Airlines wanted to advertise its new leather first‐class seats to Mexico, its
“Fly in Leather” campaign would have literally translated to “Fly Naked” in Spanish.
Ethics in Data Collection
• Treating the information given by the respondent as strictly confidential and guarding his or her
privacy is one of the primary responsibilities of the researcher.
• Personal or seemingly intrusive information should not be solicited, and if it is absolutely necessary
for the project, it should be tapped with high sensitivity to the respondent, offering specific reasons.
• Whatever the nature of the data collection method, the self‐esteem and self‐respect of the subjects
should never be violated.
• No one should be forced to respond to the survey and if someone does not want to avail
themselves of the opportunity to participate, the individual’s desire should be respected (use
informed consent).
• Nonparticipant observers should be as unintrusive as possible.
• Posting invitations to participate in a survey on social networks, discussion groups, and chat rooms is
often perceived as “spam”. Make sure that you are familiar with, and that you act in accordance with,
anti‐spam legislation and guidelines.
• There should be absolutely no misrepresentation or distortion in reporting the data collected during
the study.
Ethical behavior of respondents
• The subject, once having exercised the choice to participate
in a study, should cooperate fully in the tasks ahead, such as
responding to a survey.
• The respondent also has an obligation to be truthful and
honest in the responses. Misrepresentation or giving
information, knowing it to be untrue, should be avoided.
Experimental Designs
❑Lab experiments:
experiments done in an artificial or contrived environment

❑Field experiments:
experiments done in the natural environment in which activities regularly take
place

❑Simulation:
experiments conducted in a specially created setting that very closely
represents the natural environment in which activities are usually carried out
Control and Manipulation
Control means we exclude the contaminating factors.
• Matching groups
to match the various groups by picking the confounding characteristics and deliberately spreading
them across groups.
• Randomization
randomly assigning members to the groups we are distributing the confounding variables among
the groups equally, that the controlled variables (age, sex, and previous experience) will have an
equal probability of being distributed among the groups.

Manipulation means that we create different levels of the independent variable to

assess the impact on the dependent variable. The manipulation of the independent
variable is also known as the treatment, and the results of the treatment are called
treatment effects.
Validity of Experiments
• Internal Validity
Determination of whether the effect is actually caused by the manipulation of
treatments and not by other, exogenous variables
• External Validity
Determination of whether the cause-and-effect relationships found in the
experiment can be generalized
• Trade‐off between internal validity and external validity:
If we want high internal validity, we should be willing to settle for lower external
validity and vice versa. To ensure both types of validity, researchers usually try first
to test the causal relationships in a tightly controlled artificial or lab setting, and
once the relationship has been established, they try to test the causal relationship in
a field experiment.
Factors affecting the validity of experiments
• History effects
• Maturation effects
• Testing effects
• Selection bias effects
• Mortality effects
• Statistical regression effects
• Instrumentation effects
History effects
Certain events or factors that
have an impact on the
independent variable–dependent
variable relationship might
unexpectedly occur while the
experiment is in progress, and this
history of events would confound
the cause‐and effect relationship
between the two variables.
This will affect internal validity.
Maturation effects
• Maturation effects:
a function of the processes – both
biological and psychological –
operating within the respondents as
a result of the passage of time.
• Examples:
growing older, getting tired, feeling
hungry, and getting bored.
• There could be a maturation
effect on the dependent variable
purely because of the passage of
time.
Testing effects
The exposure of participants to the pretest may affect both the internal and
external validity of the findings.
1. A main testing effect occurs when the prior observation (the pretest) affects
the later observation (the posttest), therefore threaten the internal validity.
2. Interactive testing effects occur when the pretest affects the participant’s
reaction to the treatment (the independent variable), therefore threaten the
external validity.
Selection bias effects
• The volunteers selected may be quite different from the others (inasmuch as they
may come from an environment of deprivation) and their responses to the
treatment might be quite different.
• Such bias in the selection of the subjects might contaminate the cause‐and‐effect
relationships and pose a threat to internal validity.
• Newcomers, volunteers, and others who cannot be matched with the control
groups pose a threat to internal validity in certain types of experiment.
Mortality effects
• The mortality or attrition of the members in the experimental or control group,
or both, as the experiment progresses.
• When the group composition changes over time across the groups, comparison
between the groups becomes difficult, because those who dropped out of the
experiment may confound the results.
Statistical regression effects
• When the members chosen for the experimental group have extreme scores on
the dependent variable to begin with.
• Those with very low scores on a variable (in this case, current sales ability) have
a greater probability of showing improvement and scoring closer to the mean on
the posttest after being exposed to the treatment [“regressing toward the mean”
(statistical regression)].
• Likewise, those with very high abilities also have a greater tendency to regress
toward the mean – they will score lower on the posttest than on the pretest.
• Threat to internal validity.
Instrumentation effects
• Might arise because of a change in the measuring instrument between pretest and
posttest, and not because of the treatment’s differential impact at the end.
• Threat to internal validity.
Experimental designs
• Quasi-experimental designs
a. Pretest and posttest experimental group design
b. Posttests only with experimental and control groups
c. Time series design
• True experimental designs
a. Pretest and posttest experimental and control group design (controlling the history,
maturation, main testing, and instrumentation effects, but not mortality effects)
b. Solomon four-group design
c. Double-blind studies
• Ex post facto designs
• Completely randomized design, randomized block design, Latin square design, the
factorial design
Quasi-experimental designs
True experimental designs
Group 3 – Group 4:
[O5 – ½ (O1 + O3)] – [O6 – ½ (O1 + O3)] =
[E + U] – U = E
Group 1 – Group 3:
[O2 – O1] – [O5 – ½ (O1 + O3)] =
[E + I + U] – [E + U] = I
Subjects should be randomly selected and randomly
assigned to groups to remove statistical regression and
selection biases.
Group 2, helps us to see whether or not history,
maturation, (main) testing, instrumentation, or regression
threaten internal validity.
Solomon four-group design Groups 1 and 3 are able to control for interactive testing
impact of the experimental treatment (E), effects that threaten the external validity.
interactive testing effects (I), and uncontrolled However, mortality (the loss of participants during the
variables (U) for each group: course of the experiment) is still a potential problem.
Group 1: O2 – O1 = E + I + U
Group 2: O4 – O3 = U In a double‐blind studies both the experimenter and the
Group 3: O5 – ½ (O1 + O3) = E + U subjects are blinded (who is given the drug and who the
Group 4: O6 – ½ (O1 + O3) = U placebo).
Major threats to validity in different experimental
designs when members are randomly selected and
assigned
Types of experimental design Major threats to validity

• Pretest and posttest with one • History, maturation, main testing,

experimental group only interactive testing, mortality
• Pretest and posttest with one • Interactive testing, mortality
experimental and one control group
• Posttests only with one experimental • Mortality
and one control group
• Solomon four‐group design • Mortality
Ethical Issues in Experimental Design
Considered unethical:
❑ Putting pressure on individuals to participate in experiments through coercion, or applying social
pressure.
❑ Giving menial tasks and asking demeaning questions that diminish participants’ self‐respect.
❑ Deceiving subjects by deliberately misleading them as to the true purpose of the research.
❑ Exposing participants to physical or mental stress.
❑ Not allowing subjects to withdraw from the research when they want to.
❑ Using the research results to disadvantage the participants, or for purposes not to their liking.
❑ Not explaining the procedures to be followed in the experiment.
❑ Exposing respondents to hazardous and unsafe environments.
❑ Not debriefing participants fully and accurately after the experiment is over.
❑ Not preserving the privacy and confidentiality of the information given by the participants.
❑ Withholding benefits from control groups.
39
Reference
• Sekaran, Bougie, 2016, Research Methods for Business, 7E.
• Cooper, Schindler, 2014, Business Research Methods, 12E.
• Saunders, Lewis, Thornhill, 2016, Research Methods for
Business Students, 7E
• Miles, Huberman, Saldana, 2014, Qualitative Data Analysis: a
Methods Sourcebook, 3E