FAR EASTERN UNIVERSITY HIGH SCHOOL
Accountancy, Business and Management
Nicanor Reyes Sr. St., Sampaloc, Manila
INQUIRIES, INVESTIGATIONS AND IMMERSION
MODULE 2:
The Pillars of Measurement: Validity and Reliability in Research Tools
Learning Objectives
At the end of the lesson, students are expected to:
1. Describe the concepts of validity and reliability;
2. Identify the different types of validity and reliability;
3. Recognize the importance of validity and reliability in research;
4. Apply the necessary steps to create a reliable and valid research tool.
Lesson 1: Defining Validity and Reliability and Their Types
The concepts of validity and reliability are fundamental in research, testing, and
assessment. Both are crucial for evaluating the quality and effectiveness of procedures,
methods, techniques, and tests. By considering them, we can determine how accurately and
consistently a measurement tool assesses what it is intended to measure.
• Reliability refers to the consistency of a measurement tool in yielding the same results
under consistent conditions.
• Validity pertains to the accuracy of a measurement tool in measuring what it is
intended to measure.
In research, particularly in quantitative studies, both validity and reliability must be carefully
considered, especially when crafting the methodology section and reporting findings.
Neglecting the aforementioned can introduce bias into the research, which may
subsequently compromise the integrity and quality of the work.
How do we assess validity and reliability?
➢ To evaluate the reliability of the work, one typically assesses the consistency of
results over time, across different observers, and across various components of the
measurement tool.
➢ To evaluate validity, one can assess how well the findings align with established
theories and other measures of the same concept.
FAR EASTERN UNIVERSITY HIGH SCHOOL
Accountancy, Business and Management
Nicanor Reyes Sr. St., Sampaloc, Manila
INQUIRIES, INVESTIGATIONS AND IMMERSION
Why do validity and reliability go hand in hand?
➢ A valid tool is considered reliable. If a measurement tool produces accurate results,
those results should be reproducible.
➢ High reliability can serve as a good indicator that validity is present. After all, if a
measurement tool is not reliable, how can it possibly be valid?
o However, a reliable instrument is not always valid. While it may consistently
produce the same results, those results are not necessarily accurate.
Take note: Validity and reliability are interconnected concepts; however, they do not share the
same meaning.
EXAMPLES
Reliability Validity
Hanna is conducting an experiment Dr. Trish used a test to diagnose a patient
using a freshwater sample. She measured with diabetes. This tool consistently
the temperature of the sample several times provided reliable diagnoses when used by
under identical conditions. She observed different doctors at various times. This
that the thermometer displayed the same consistency indicates that the test has high
results each time. Thus, she concluded that validity as a measurement tool for
the results are reliable. (WITH diagnosing diabetes. (WITH VALIDITY)
RELIABILITY)
Several ABM students produced a
Despite Hanna carefully controlling the measurement tool designed to assess
conditions to ensure the sample's endurance. While the results yielded are
temperature remained constant, the reliable, the scores are strongly linked to
thermometer displayed different agility rather than endurance. This indicates
temperatures each time. (LOW that the method has low validity as it does
RELIABILITY) not accurately measure the intended
attribute. (LOW VALIDITY)
Considering these examples, we can infer that assessing validity is often more challenging
compared to reliability. Nonetheless, ensuring the validity of measurement tools is crucial
for obtaining accurate results, which, in turn, facilitates meaningful and valid discussions and
conclusions.
FAR EASTERN UNIVERSITY HIGH SCHOOL
Accountancy, Business and Management
Nicanor Reyes Sr. St., Sampaloc, Manila
INQUIRIES, INVESTIGATIONS AND IMMERSION
Types of Reliability
Types Description
Test-retest Reliability - The consistency of a tool over time.
Guide Question for the researcher: Do you obtain the same
results when repeating the measurement?
Example:
Section 1 completed a test to measure their IQ. If the students
repeat the test weeks or months later and achieve similar
scores, this indicates high test-retest reliability.
Interrater Reliability - The consistency of a tool across different observers or
(also known as raters.
interobserver reliability)
Guide Question for the researcher: Do you obtain consistent
results when different individuals perform the same
measurement?
Example:
Lemuel is conducting a study where he invited two teachers to
independently grade the essays of 20 students. They used a
standardized rubric with criteria such as clarity, organization,
and content, each rated on a scale of 1 to 5. If both teachers give
similar scores to the students, this indicates high interrater
reliability. This consistency suggests that the rubric is a reliable
tool, regardless of who the observer is.
Internal Consistency - The consistency of the measurement itself.
Guide Question for the researcher: Do different parts of the tool
designed to measure the same thing yield consistent results?
Example:
Murphy aims to measure the customer satisfaction of Gen Zs
with the XYZ fast food chain. He created a questionnaire with a
set of statements that respondents might agree or disagree
with. The choices are: strongly agree, agree, neutral, disagree,
and strongly disagree. If the respondents provide consistent
answers across each item, this indicates a high level of internal
consistency. However, if the answers vary significantly, this
suggests that the questions may be poorly worded and are not
reliable in measuring customer satisfaction.
FAR EASTERN UNIVERSITY HIGH SCHOOL
Accountancy, Business and Management
Nicanor Reyes Sr. St., Sampaloc, Manila
INQUIRIES, INVESTIGATIONS AND IMMERSION
Types of Validity
Types Description
Construct Validity - The alignment of a measurement tool with existing theory
and established knowledge of the concept being
measured.
Types of Construct Validity:
1. Convergent Validity: The degree to which your measure
aligns with or correlates to measures of related
constructs.
2. Discriminant Validity: The degree to which your measure
is uncorrelated or negatively correlated with measures of
distinct constructs.
Remember: If your constructed questions address constructs
other than your intended one, it suggests that the questionnaire
is too broad and should be refined to focus primarily on your
specific construct.
Content Validity - The degree to which the instrument encompasses all
relevant aspects of the concept being evaluated.
Guide Question for the researcher: Does the tool comprehensively
measure all facets of the construct I aim to assess?
Remember: The greater the content validity, the more accurately
the measurement reflects the chosen concept or construct.
Face Validity - This type of validity evaluates whether the tool seems
relevant and appropriate for its intended purpose at face
value. It serves as the initial step in assessing the overall
validity of the measurement tool.
Guide Question for the researcher: Does the content of the tool
appear to be aligned with its aims?
Remember: Good face validity implies that anyone assessing the
tool will conclude it appears to measure what it is supposed to
measure. In contrast, if the tool exhibits poor face validity, the
evaluator will likely be confused about what you are measuring.
Criterion Validity - The degree to which the results of a measure align with
(also known as those of other valid measures of the same concept.
FAR EASTERN UNIVERSITY HIGH SCHOOL
Accountancy, Business and Management
Nicanor Reyes Sr. St., Sampaloc, Manila
INQUIRIES, INVESTIGATIONS AND IMMERSION
criterion-related Types of Criterion Validity:
validity) 1. Concurrent Validity: The evaluation of a test's scores by
comparing them with criterion variables that are
measured simultaneously.
→ Example: Test score and actual diagnosis that was
obtained at the same time
2. Predictive Validity: is assessed when the criterion
variables are measured after the test scores have been
obtained.
→ Example: College entrance exam and GPA after the 1st
semester
Take note: This type of validity is often employed when a
researcher seeks to replace an established test that is considered
lengthy with a shorter version of the same test.
If the scores from two different tools are similar (i.e., the old and
new), the newly developed tool demonstrates high criterion
validity.
Lesson 2: Steps in Developing a Research Instrument
Questionnaires are the most frequently used data collection method in educational and
evaluation research. They help gather information on knowledge, attitudes, opinions,
behaviors, facts, and more. Developing a valid and reliable questionnaire is crucial to
reducing measurement errors. According to Groves (1987), measurement error is defined as
the “discrepancy between respondents' attributes and their survey responses.”
The following are the necessary steps to craft a good research instrument:
1. Background
• In the first step, the purpose, objectives, research questions, and hypothesis of the
proposed study are taken into account. This step also involves identifying the
audience, their background (particularly their educational and reading levels),
accessibility, and the method used to choose the respondents (sample vs. population).
A deep understanding of the issue through readings and a review of the literature is
essential. Step 2 can only commence with adequate preparation and understanding
of Step 1.
FAR EASTERN UNIVERSITY HIGH SCHOOL
Accountancy, Business and Management
Nicanor Reyes Sr. St., Sampaloc, Manila
INQUIRIES, INVESTIGATIONS AND IMMERSION
2. Questionnaire Conceptualization
• Once you have a solid understanding of the research, the next step is to create
statements or questions for the questionnaire. This phase involves transforming the
content from the theoretical framework or literature into questions or statements.
Furthermore, a connection is made between the objectives of the research and its
translation into the content. For instance, the researcher needs to specify what the
questionnaire is intended to measure—knowledge, attitudes, views, opinions, fact
recall, behavior modification, etc. This stage involves defining and identifying the
major variables: independent, dependent, and moderator variables.
3. Format & Data Analysis
• Step 3 focuses on creating statements and questions, choosing suitable measurement
scales, organizing the questions in the questionnaire, determining the format and font
size, designing both front and back covers, and proposing data analysis methods.
• Scales are tools for measuring a subject's response to a certain variable.
• It is critical to understand how the appropriateness of data analysis and the
measurement level relate to one another. For instance, if ANOVA (analysis of
variance) is one method of data analysis, the dependent variable must be assessed on
an interval/ratio scale (strongly agree to strongly disagree), and the independent
variable must be tested on a nominal scale with two or more levels (yes, no, not sure).
4. Establishing Validity
• Steps 1-3 have produced a draft questionnaire that is now prepared for validity
testing. Validity is determined through a field test and a panel of experts. The study's
goals will determine which form of validity to employ—content, concept, criteria, or
face validity.
• In Step 4, the following queries are addressed:
o Is the questionnaire valid? In other words, does it measure what it is intended to
measure?
o Does it accurately represent the content?
o Is it appropriate for the target sample or population?
o Is the questionnaire comprehensive enough to gather all the necessary
information to address the study's purpose and goals?
o Does the instrument resemble a questionnaire in both appearance and format?
FAR EASTERN UNIVERSITY HIGH SCHOOL
Accountancy, Business and Management
Nicanor Reyes Sr. St., Sampaloc, Manila
INQUIRIES, INVESTIGATIONS AND IMMERSION
5. Establishing Reliability
• The final phase assesses the questionnaire's dependability using a pilot test. According
to Norland (1990), the measuring instrument's accuracy or precision is indicated by
its reliability. The pilot test aims to address whether the questionnaire consistently
measures what it is intended to measure.
• Depending on the type of data (nominal, ordinal, interval/ratio), different types of
reliability (test-retest, split-half, alternate form, and internal consistency) are used.
For instance, internal consistency is a suitable tool to evaluate the dependability of
questions on an interval or ratio scale. Test-retest or split-half methods are
appropriate for evaluating the reliability of knowledge questions.
If there is an existing research instrument that can be used in your study, make sure to ask
for permission to use it. The request should be addressed to the publisher's permissions
department or, if applicable, the specific copyright holder. Ensure your letter or message
includes the date, your phone and fax numbers, your email address, and your return address
at the top. However, if the study indicates that the research instrument is available for public
use, make sure to provide proof of this and attach it to your study's appendices.
