Quantitative Framework 2023
Uploaded by
Rashel ten HolderQuantitative Framework 2023
Uploaded by
Rashel ten HolderCritical Evaluation of Research Papers – RCTs/Experimental Studies
Concept Comment
What is the objective for the research?
P (Patient) -
I (Intervention) -
C (Comparison) -
O (Outcome) –
INTERNAL VALIDITY
1. Randomisation
• How were participants allocated to each
group?
• Did a randomisation process taken place?
2. Comparability of Groups
• How similar were the groups? Eg age,
sex, ethnicity – is this made clear?
3. Blinding (none, single, double or triple)
• Who was not aware of which group a
patient was in? (eg nobody, only patient,
patient and clinician, patient, clinician and
researcher?)
• Was it feasible for more blinding to have
taken place?
4. Equal treatment of groups
• Were both groups treated in the same
way?
1
5. Attrition
• What percentage of participants dropped
out? Did this adversely affect one
group?
• Has this been evaluated?
Overall Internal Validity
(Does the research measure what it is
supposed to be measuring?)
EXTERNAL VALIDITY
6. Attrition
• Was everyone accounted for at the end of
the study?
• Was any attempt made to contact drop-
outs?
7. Sampling approach
• How was the sample selected?
• Was it based on probability or non-
probability?
• What was the approach (eg simple
random, convenience)?
• Was this an appropriate approach?
8. Sample size (power calculation)
• How many participants?
• Was a sample size calculation performed?
• Did the study pass?
9. Exclusion/inclusion criteria
• Were the criteria set out clearly?
• Were they based on recognised diagnostic
criteria?
2
Overall External Validity
(Can the results be applied to the wider
population?)
RELIABILITY (MEASUREMENT TOOL) – internal validity
10. Internal consistency reliability
(Cronbach’s Alpha)
• Has a Cronbach’s Alpha score of 0.7 or
above been included?
11. Test re-test reliability correlation
• Was the test repeated more than once?
• Were the same results received?
• Has a correlation coefficient been
reported? Is it above 0.7?
12. Validity of measurement tool
• Is it an established tool?
• If not what has been done to check if it is
reliable?
• Pilot study
• Expert panel
• Literature review
• Criterion validity (test against other
tools)
o Has a criterion validity
comparison been carried out?
Was the score above 0.7?
Overall Reliability
(How consistent are the measurements?)
Overall, how valid and reliable is the paper?
3
References:
LoBiondo-Wood, G. and Haber, J. (2014) ‘Reliability and validity’, in LoBiondo-Wood, G. and Haber,
J. (eds.) Nursing research. 8th edn. St. Louis: Mosby Elsevier, pp.289-309.
Taylor, M.C. (2007) Evidence-based practice for occupational therapists. 2nd edn. Oxford:
Blackwell Publishing.