Research Series Article

How to Write a Systematic Review Without Meta‑analysis

Nishad Plakkal
Downloaded from http://journals.lww.com/iamr by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

Department of Neonatology, JIPMER, Puducherry, India

Abstract
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8KKGKV0Ymy+78= on 04/21/2025

Systematic reviews (SRs) are a way of synthesizing evidence from research and are extensively used in health care to inform clinical practice,
research, and policy. In contrast to narrative reviews, they tend to be more rigorous and reproducible, with the added option of statistically
synthesizing the results in a meta‑analysis. SRs have historically focused on interventional studies; however, SRs currently cover a wide
variety of study designs. The team conducting the review usually includes a librarian in addition to researchers. The first step of the review
is formulating a focused research question. After registering the protocol, the team systematically searches medical databases for relevant
literature using a predefined search strategy. Software tools and web‑based platforms such as Covidence and Rayyan are available to assist
researchers with the process of screening citations and managing the whole review workflow. Ineligible studies are discarded after screening.
Eligible studies are then carefully assessed for quality and risk of bias (ROB) using quality assessment tools, like the Cochrane ROB tool (for
randomized controlled trials). Once the data are extracted and summarized, the reviewers interpret the findings and prepare a report. The
Preferred Reporting Items for SRs and Meta‑analyses statement, last updated in 2020, is specifically designed to improve the quality of
reporting of SRs. Scoping reviews, umbrella reviews, rapid reviews, and living reviews are relatively newer concepts related to SRs; these
are beyond the scope of this article.

Keywords: Evidence, literature search, synthesis, systematic review

Introduction instance, about the effectiveness of an intervention or the

effect of a risk factor or an outcome. Narrative reviews can
One of the paradoxes in medicine is that the exponential
be very useful for outlining a broad perspective. However,
increase in evidence generated from research in the last few
since there is no transparent search strategy or objective
decades has made the interpretation and application of the
synthesis of evidence from the retrieved articles, such reviews
evidence more difficult. The difficulty arises mainly from
can be potentially biased, colored by the author’s opinions
two factors: the sheer number of studies and often conflicting
and preferences. On the other hand, an SR, as the name
results from individual studies. Systematic reviews (SRs) aim
indicates, is done in a systematic and reproducible fashion.
to bridge the gap between evidence and practice by providing
SRs aim to present the reader with theoretical conclusions
objective summaries of research findings related to a particular
based on a comprehensive, reproducible search and objective
topic.[1] Since they synthesize and interpret the evidence
assessment of evidence related to a specific topic. Individuals
from multiple studies, they are considered high‑quality
and organizations can then use the reviewers’ conclusions to
evidence and sit at (or near) the top of the evidence pyramid.
optimize clinical practice, generate guidelines, frame health
Although not discussed in detail in this article, it may also be
policies, prioritize research ideas, or allocate funds rationally.
possible to generate a quantitative statistical summary of the
Although historically focused on interventional studies,
results from included studies using a meta‑analysis. When
especially randomized controlled trials, the current SRs cover
correctly done, SRs, especially with a meta‑analysis, generate
stronger evidence than individual studies, particularly smaller
Address for correspondence: Dr. Nishad Plakkal,
observational studies or randomized trials.[1,2] Department of Neonatology, JIPMER, Puducherry ‑ 605 006, India.
E‑mail: [email protected]
When summarizing existing evidence about a topic or research
question, the objectives are often to give the readers a broad
perspective as well as specific, reproducible answers – for This is an open access journal, and articles are distributed under the terms of the Creative
Commons Attribution‑NonCommercial‑ShareAlike 4.0 License, which allows others to
Access this article online remix, tweak, and build upon the work non‑commercially, as long as appropriate credit
is given and the new creations are licensed under the identical terms.
Quick Response Code:
Website: For reprints contact: [email protected]
www.ijamhrjournal.org
How to cite this article: Plakkal N. How to write a systematic review
without meta‑analysis. Int J Adv Med Health Res 2023;10:119-23.
DOI:
10.4103/ijamr.ijamr_296_23 Submitted: 21‑10‑2023, Revised: 06‑11‑2023,
Accepted: 11‑11‑2023, Published: 15-12-2023

© 2023 International Journal of Advanced Medical and Health Research | Published by Wolters Kluwer - Medknow 119
 Plakkal: How to write a systematic review

a wide variety of study designs, including observational and anywhere around the world. It is good practice to register
diagnostic studies.[3] the SR protocol to ensure transparency. This must be done
before extracting any data. The most common site for review
Planning and Conducting a Systematic Review registration is the International Prospective Register of
Systematic Reviews (PROSPERO; https://www.crd.york.
Irrespective of the type of studies being reviewed, there ac.uk/PROSPERO/). Some journals also offer peer review
are common steps involved in conducting an SR without a and publication of SR protocols: for example, trials (https://
Downloaded from http://journals.lww.com/iamr by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

meta‑analysis.[2,4] These steps are covered in the Preferred trialsjournal.biomedcentral.com/) and PLOS (https://plos.org/
Reporting Items for SRs and Meta‑Analyses (PRISMA) protocols/).
statement and are discussed below.[5,6] Figure 1 is a visual
summary of these steps. Define the research question
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8KKGKV0Ymy+78= on 04/21/2025

A clearly formulated research question is at the heart of

Choose the team and the tools the review. Fine‑tuning the research question involves a
Clearly, the team should include researchers with expertise in preliminary search of the literature (scoping) to identify
the topic. In addition, the team usually includes a librarian and knowledge gaps and check for recent SRs related to the
someone with experience in SR methodology.[2,4] Any conflicts question in your mind. A search on PROSPERO might also
of interest should be declared explicitly – for example, a reveal reviews on the same topic that are under progress or
financial interest in a drug or other intervention. The librarian is have been completed but not published. For studies involving
someone with experience in searching for literature. Depending intervention, the question is often written in the Population,
on the type of review, other team members may be involved, Intervention, Comparison and Outcome (PICO) format. For
for example, an economist for reviews, including economic example, pregnant patients with gestational diabetes were
analysis. It is important to remember that SRs are generally often unable to visit the hospital during the recent COVID‑19
time‑consuming, often taking 1–2 years to complete; hence, pandemic, and many doctors resorted to telemedicine to
the availability of the team members should be confirmed in manage them. A resident, looking back to see how effective
advance. telemedicine was for this condition, could just type the key
Since the team will have to collaborate extensively throughout phrases into popular search engines like Google. If they want
the process of the review, it is important to decide in advance to search the literature more objectively, the question can be
what tools or platforms to use for the review. Online broken down using the PICO format as follows:
collaboration has been made easy by the emergence of Among pregnant women with gestational diabetes (Population),
tools such as Covidence (https://www.covidence.org/) and do telemedicine consultations (Intervention), when compared
Rayyan (https://www.rayyan.ai/). Cochrane reviews can be to in‑person consultations (Comparison), improve blood sugar
done using RevMan Web (https://revman.cochrane.org/).[7] control (Outcome)?
RevMan 5, the older Cochrane software for reviews, is no
Similar frameworks are available for other types of studies:
longer being maintained but is still available and is a very
for example, Sample, Phenomenon of Interest, Design,
capable tool. These tools assist and enhance the SR workflow,
Evaluation, Research Type for qualitative or mixed‑methods
with the ability to import citations, eliminate duplicates,
studies. These help to make the question more objective and
screen abstracts, extract data, prepare risk of bias (ROB)
narrow down the search terms. A common strategy is to then
tables, and collaborate with team members working from
search for the individual elements of the question (and their
synonyms) in databases like PubMed and then combine the
results using Boolean operators (AND, OR, and NOT; only the
first two are usually used). A hypothetical list of search terms
in the PICO framework, combined using Boolean operators,
is shown below:
1. P: Gestational diabetes OR (pregnancy AND diabetes)
AND
2. I: Telemedicine OR telehealth OR remote consultation
OR mobile apps
AND
3. C: In‑person consultation OR in‑person visits
AND
4. O: Glucose control.
Search for relevant research
This requires careful planning of the search strategy. At the
Figure 1: Steps involved in a systematic review. Although not shown outset, the team decides if they want to conduct a SR from
here, it is the best practice to register (and publish) the review protocol scratch or just update a SR by adding newer studies. The

120 International Journal of Advanced Medical and Health Research ¦ Volume 10 ¦ Issue 2 ¦ July‑December 2023
 Plakkal: How to write a systematic review

latter approach is appropriate when the team has access to abstracts, this task may have to be split up between multiple
a high‑quality SR conducted earlier and saves considerable team members. The reasons for the exclusion of articles should
time and effort.[8] The Cochrane Collaboration recommends be documented. A checklist usually helps with this process.
that the literature search should be conducted by librarians (or Citation software or online platforms like Rayyan help to
someone with extensive literature search experience). Two identify and remove duplicate citations. While an Excel sheet
or more team members are usually involved in the search. is a low‑tech way of tracking this process, platforms like
The team needs to decide which databases they will access Covidence make it much easier to manage the workflow.
Downloaded from http://journals.lww.com/iamr by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

to conduct the search. Most authorities recommend searching

three or more databases. A good starting point is to search
Extract and summarize data
This is usually a time‑consuming step, although the tools
repositories of SRs, like the Cochrane Central Register of
mentioned earlier have begun to make this process easier. Even
Controlled Trials (CENTRAL; https://www.cochranelibrary.
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8KKGKV0Ymy+78= on 04/21/2025

if a meta‑analysis is not planned, extracted data are required to

com/central). For medical research, we typically search
make conclusions about the outcome, effect size, cost, safety,
PubMed (or MEDLINE; https://pubmed.ncbi.nlm.nih.gov/),
and other key factors that influence the final recommendations.
Embase (https://www.embase.com/), and Google Scholar as
At least two reviewers should independently extract data
well, but depending on the research question and the resources
using identical forms or tools.[11] This helps to minimize
available; other databases such as Cumulative Index to Nursing
errors, particularly with respect to key outcomes. If agreement
and Allied Health Literature, PsycINFO, and Web of Science
between the reviewers is to be formally assessed, kappa
may also be involved.
statistics can be used. The protocol should also specify the
The keywords and index terms used are noted down in a steps to address any conflict that cannot be resolved – usually
document, usually divided by the components of the research by consensus or by arbitration by a third person. If important
question (population, intervention, and outcome). Possible information is missing, every effort is made to obtain this from
synonyms for relevant search terms are carefully considered other sources (e.g., overlapping publications and conference
and included (e.g., “neonates” and “infants”). The search presentations) or by contacting the author(s) directly. Some of
strategy used for at least one database is included in the the important elements of data collection are listed in Table 1.
methods section of the published review; others may be given
in the supplementary materials if the journal permits it. In Assess quality and risk of bias
addition to searching these databases, reviewers may decide to This is a key step in the process of SR. Quality and ROB
search relevant conference proceedings, government reports, are assessed separately for each included study by two
dissertations, or other nontraditional sources of medical team members working independently, and the findings are
literature, including preprint or unpublished studies. These pooled. The Cochrane ROB 2.0 Tool is an example of a tool
sources are often collectively called the “gray literature.”[9] that provides templates to assess ROB in randomized trials.
It is worth noting that Google Scholar supports Boolean
operators, and since it searches the full text of articles, books, Table 1: A partial list of information that goes into the
and conference proceedings from more sources than PubMed data collection tool or proforma
or Embase, it is a good search engine for gray literature. Basic information
However, the searches are not reproducible, and the number Study ID, name of data extractor, and date of data extraction
of articles retrieved may be very large.[10] Depending on the Eligibility
scope of the review, the articles retrieved may be only a few Reason for inclusion/exclusion
or may run into thousands. Methods
Study design, single center or multicentric, recruitment procedures,
Select eligible studies randomization details, allocation concealment, masking (for
Inclusion and exclusion criteria are based on study design randomized controlled trials), methods used to prevent or control
type, study population, and reported outcomes and are defined confounding, biases, missing data and how it was handled, statistical
at the outset by the team.[2,4] Inclusion criteria are designed methods
Setting and participants
to include the maximum number of articles for screening so
Details of intervention and comparison
that no relevant article is missed. It is a good practice to keep
e.g., dose, frequency, specific procedures, and length of exposure
exclusion criteria to the minimum. Although we commonly
Outcomes
include only English‑language articles, this is a choice based Including measurement tools and instruments; adverse outcomes
on practical considerations and is actually a limitation of the Results
review. Number in each group, measurements of outcomes with their
units, summary measures such as means and standard deviations,
These criteria are then used to screen and select (or reject) between‑group comparisons including risk ratios and ORs
each identified article. This is typically a two‑ or three‑step Comments
process. Many articles can be excluded after reading the title Missing data, need for correspondence with the authors
and abstract. Reading the full text may help exclude more. The exact information collected will depend on the study type and the
Depending on the scope of the review and the number of objectives of the review. ORs=Odds ratios

International Journal of Advanced Medical and Health Research ¦ Volume 10 ¦ Issue 2 ¦ July‑December 2023 121
 Plakkal: How to write a systematic review

Although not in the scope of this article, the estimated effect clinicians, researchers, funders, governments, and other end
sizes from the included studies can be plotted against the users. In the context of increasing research output, often
study’s size or precision to create a scatter plot known as the with conflicting results, SRs help to summarize and clarify
funnel plot. the evidence. With the availability of software tools, high
levels of automation are now possible, making it easier than
Tools are also available to assess the quality of studies,
ever to conduct SRs. Some useful resources are listed in the
depending on the type of study.[12,13] Checklists from the
Appendix 1 for those interested in conducting SRs.
Downloaded from http://journals.lww.com/iamr by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

Critical Appraisal Skills Program (CASP) are available for

different study designs, for example, CASP for cohort studies. Financial support and sponsorship
These tools help us to objectively measure our confidence Nil.
in the quality of the studies and their results, which in turn
Conflicts of interest
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8KKGKV0Ymy+78= on 04/21/2025

determine the strength of the recommendations, particularly

There are no conflicts of interest.
if the SR is being conducted to create a guideline. Grading
of Recommendations, Assessment, Development, and
Evaluation (GRADE; https://www.gradeworkinggroup.org/) References
is a system that is commonly used to assess the certainty of 1. Mulrow CD. Rationale for systematic reviews. BMJ 1994;309:597‑9.
2. Siddaway AP, Wood AM, Hedges LV. How to do a systematic review:
evidence and determine the strength of recommendations in A best practice guide for conducting and reporting narrative reviews,
health‑care guidelines. meta‑analyses, and meta‑syntheses. Annu Rev Psychol 2019;70:747‑70.
3. Iddagoda MT, Flicker L. Clinical systematic reviews – A brief overview.
Report and disseminate results BMC Med Res Methodol 2023;23:226.
The PRISMA statement provides the most commonly used 4. Muka T, Glisic M, Milic J, Verhoog S, Bohlius J, Bramer W, et al.
reporting standards for SRs.[5,6] It is important to keep the A 24‑step guide on how to design, conduct, and successfully publish
a systematic review and meta‑analysis in medical research. Eur J
big picture in mind when reporting SRs: The idea is not Epidemiol 2020;35:49‑60.
just to tabulate the information; authors must also use 5. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC,
their expertise to reflect on the evidence and offer critical Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline
for reporting systematic reviews. BMJ 2021;372:n71.
insights that improve our understanding of the literature.
6. Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D,
Like many other research articles, SRs have five sections: Page MJ, et al. PRISMA‑S: An extension to the PRISMA statement for
introduction, methods, results, discussion, and conclusion. reporting literature searches in systematic reviews. J Med Libr Assoc
A structured abstract is also provided. The title explicitly 2021;109:174‑200.
7. Cowie K, Rahmatullah A, Hardy N, Holub K, Kallmes K. Web‑based
states that it is an SR (and meta‑analysis, if it is included). software tools for systematic literature review in medicine: Systematic
Funding sources, if any, must be disclosed. The flow of search and feature analysis. JMIR Med Inform 2022;10:e33219.
studies through the SR can be depicted using the PRISMA 8. Garner P, Hopewell S, Chandler J, MacLehose H, Schünemann HJ,
2020 flow diagram. Increasingly, a plain language summary Akl EA, et al. When and how to update systematic reviews: Consensus
and checklist. BMJ 2016;354:i3507.
is also included with the SR. A variety of journals accept 9. Nair A, Borkar NK. Significance of including grey literature search in
SRs for publication. Just as for other research, wider systematic reviews and meta‑analyses. Saudi J Anaesth 2023;17:295‑6.
dissemination through professional organizations or social 10. Haddaway NR, Collins AM, Coughlin D, Kirk S. The role of google
scholar in evidence reviews and its applicability to grey literature
media helps to disseminate the information to a wider
searching. PLoS One 2015;10:e0138237.
audience. 11. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al
(editors). Cochrane handbook for systematic reviews of interventions
The landscape of SRs has been changing rapidly. The recent version 6.4. Cochrane 2023. Available from: www.training.cochrane.
pandemic resulted in an unprecedented deluge of information org/handbook. [Last updated on 2023 Aug 01].
related to various aspects of COVID‑19. This highlighted the 12. Ma LL, Wang YY, Yang ZH, Huang D, Weng H, Zeng XT.
need for constantly updated, “living” SRs.[14] With the advent Methodological quality (risk of bias) assessment tools for primary and
secondary medical studies: What are they and which is better? Mil Med
of artificial intelligence and sophisticated software tools, it Res 2020;7:7.
is now feasible to have SRs that are updated in real time or 13. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND,
at least very rapidly. These are online, continuously updated Viswanathan M, et al. ROBINS‑I: A tool for assessing risk of bias in
non‑randomised studies of interventions. BMJ 2016;355:i4919.
documents, unlike traditional SRs which get outdated as new 14. Heron L, Buitrago‑Garcia D, Ipekci AM, Baumann R, Imeri H,
evidence comes in.[15,16] Unlike the current SRs which take Salanti G, et al. How to update a living systematic review and keep it
months or even years to plan, conduct, and publish, living SRs alive during a pandemic: A practical guide. Syst Rev 2023;12:156.
rely on automation to identify studies, extract data, and revise 15. Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S,
et al. Living systematic review: 1. Introduction‑the why, what, when,
the conclusion(s) rapidly. and how. J Clin Epidemiol 2017;91:23‑30.
16. Klerings I, Robalino S, Booth A, Escobar-Liquitay CM, Sommer I,
In summary, SRs are a form of evidence synthesis using
Gartlehner G, et al. Cochrane rapid reviews methods group. Rapid
objective and reproducible methods. If conducted rigorously, reviews methods series: Guidance on literature search. BMJ Evid Based
SRs provide high‑quality evidence that can be used by Med 2023;28:412-7.

122 International Journal of Advanced Medical and Health Research ¦ Volume 10 ¦ Issue 2 ¦ July‑December 2023
 Plakkal: How to write a systematic review

Appendix
Appendix 1: Useful resources for planning and conducting a systematic review
• PRISMA statement: The 2020 statement, including checklists and study flow diagram, is available at the PRISMA website:
http://www.prisma-statement.org/PRISMAStatement/
• Cochrane Central: Useful to search for randomized controlled trials related to the review topic: https://www.cochranelibrary.
com/central/about-central
Downloaded from http://journals.lww.com/iamr by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AW

• Clinical Trial Registries: To access information on unpublished or ongoing trials. Examples include the Clinical Trials
Registry‑India (https://ctri.nic.in/Clinicaltrials/login.php) and the WHO Trials Registry (https://www.who.int/clinical-
trials-registry-platform)
• Cochrane Database of Systematic Reviews: Lists all published Cochrane reviews (https://www.cochranelibrary.com/cdsr/
nYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8KKGKV0Ymy+78= on 04/21/2025

table-of-contents)
• Campbell Collaboration: They are a nonprofit organization focused on social science research, using systematic reviews (SR)
for evidence synthesis (https://www.campbellcollaboration.org/)
• Cochrane Handbook for Systematic Reviews of Interventions: An excellent resource for researchers, particularly if they
are new to the field of SRs (https://training.cochrane.org/handbook/current)
• Harvard Library: For a short summary and some useful tips about SRs and meta‑analyses (https://guides.library.harvard.
edu/meta-analysis)
• The CDC Guide to SRs: (https://www.cdc.gov/library/researchguides/systematicreviews.html)
• Rayyan: For collaborative management of the SR process. Particularly useful to screen articles and extract data (https://
www.rayyan.ai/)
• Covidence: A very popular application for managing SRs (https://app.covidence.org/).

International Journal of Advanced Medical and Health Research ¦ Volume 10 ¦ Issue 2 ¦ July‑December 2023 123