Research Methods
Session 4
By Ehsan Zolfaghari Younesi
[email protected] [email protected] Chapter 9: Research Instruments
Researchers use different instruments for collecting data.
The instruments help them test the hypothesis or
hypotheses they have formulated or the research
question(s) they intend to investigate. The present
chapter introduces different tools or instruments that
can be used in a qualitative or quantitative study.
Test
Tests are the most commonly used instruments for data collection.
Different types of tests such as language proficiency tests are usually
employed to examine whether participants are at the same level of
language proficiency before a study.
Tests administered at the outset of the study would enable
researchers to attribute any future changes in the participants'
language behavior to the manipulation of the independent variable.
Achievement tests are also used to examine the extent to which
learners have learned after a period of exposure to instruction.
Researchers should ensure that the tests they use have validity,
reliability, and suitability.
Rating Scales (I)
Multiple-choice tests eliminate the need to make subjective
judgments about an individual's performance in a particular
area.
However, occasionally researchers intend to measure the
performance of an individual or a group of individuals in areas
that, by nature, are subjective. Therefore, they need to transform
the qualitative data into quantitative measures. In these cases,
the use of rating scales seems to be necessary.
Rating Scales (II)
However, the characteristics and qualifications of the raters are also
crucial in the process of rating since they can affect the scoring
procedure. Therefore, usually, two raters (using the same rating scale)
should score students' performances on areas like speaking and writing
that are subjective.
The correlation of the two sets of scores given by the two raters is called
inter-rater reliability. Inter-rater reliability is the degree of consistency
between two raters. It is also possible to have one rater to score the
students' performances twice, which is called intra-rater reliability. In
doing so, a one-week interval is necessary.
Questionnaire
When the phenomena under investigation are not directly observable
(e.g., feelings, attitudes, perceptions, and beliefs), researchers can use
questionnaires for data collection.
Questionnaires help researchers elicit data from the participants of a
study. However, their use for data collection should be done carefully,
since many respondents may feel reluctant to respond honestly and
accurately.
Moreover, they may be conservative and answer in a way to satisfy the
researcher or consider the socially accepted norms (Halo effect). The
respondents' incorrect answers to a questionnaire can affect the results
of a study.
Interview
Interviewing, as a method of data collection, is
useful when researchers deal with qualitative
studies. Depending on the purpose of the research,
interviews can substitute questionnaires for
collecting data about individuals' views,
experiences, and attitudes.
Observation
Observation, similar to questionnaire and interview, is used
for data collection in TEFL.
Observation, in TEFL, refers to a systematic description of
students' or teachers' behavior in classroom settings. It
enables researchers to study participants' activities in natural
settings and analyze the context or phenomenon under
investigation.
Typically, it is appropriate to combine observation with other
types of data collection procedures.
Factual information (actors, activities, goal, behavior)
Interactions and quotations (summaries of conversations, patterns of
specific behavioral events, such as, conflicts, decision-making, or
collaboration, describe the participants and their roles, non-verbal
communication, the meaning of what was observed from the perspectives
of the participants, Record exact quotes or close approximations of
comments that relate directly to the purpose of the study, describe any
impact you might have had on the situation)
Reflective Content (personal responses, feelings, impressions, thoughts,
criticisms, questions, concerns, why you believe specific phenomenon
occurred)
Other Instruments
Depending on the type of the data, a researcher can use
a variety of instruments. Videotapes, audiotapes,
portfolios, diaries, students' projects and performances
are the most frequent ones for data collection.
