Mahdi et al.

BMC Emergency Medicine (2023) 23:52 BMC Emergency Medicine

https://doi.org/10.1186/s12873-023-00824-8

RESEARCH Open Access

Systematic review on the current state

of disaster preparation Simulation Exercises
(SimEx)
Syed Sarosh Mahdi1,2*, Hafsa Abrar Jafri1, Raheel Allana3, Gopi Battineni4, Mariam Khawaja1, Syeda Sakina5,
Daniyal Agha1, Kiran Rehman6 and Francesco Amenta4

Abstract
Introduction The simulation exercise (SimEx) simulates an emergency in which an elaboration or description of
the response is applied. The purpose of these exercises is to validate and improve plans, procedures, and systems
for responding to all hazards. The purpose of this study was to review disaster preparation exercises conducted by
various national, non-government, and academic institutions.
Methodology Several databases, including PubMed (Medline), Cumulative Index to Nursing and Allied Health
Literature (CINAHL), BioMed Central, and Google Scholar, were used to review the literature. Information was retrieved
using Medical Subject Headings (MeSH) and documents were selected according to Preferred Reporting Items for
Systematic Reviews and Meta-Analyses (PRISMA). To assess the quality of the selected articles, the Newcastle-Ottawa
Scale (NOS) technique was utilized.
Results A total of 29 papers were selected for final review based on PRISMA guidelines and the NOS quality
assessment. Studies have shown that many forms of SimEx commonly used in disaster management including
tabletop exercises, functional exercises, and full-scale exercises have their benefits and limitations. There is no
doubt that SimEx is an excellent tool for improving disaster planning and response. It is still necessary to give SimEx
programs a more rigorous evaluation and to standardize the processes more thoroughly.
Conclusions Drills and training can be improved for disaster management, which will enable medical professionals
to face the challenges of disaster management in the 21st century.
Keywords Disaster preparedness, Emergency response, SimEx, Mass Casualty Exercise (MCI), Disaster drills

3
*Correspondence: Department of Paediatrics & Child Health, Aga Khan University Karachi,
Syed Sarosh Mahdi Karachi 74800, Pakistan
4
[email protected] Clinical research centre, School of Medicinal and Health Products
1
Jinnah Medical and Dental College, Department of Community Sciences, University of Camerino, Camerino 62032, Italy
5
Dentistry, Sohail University, Karachi, Pakistan Sociology department, University of North Texas, Denton, TX76203, USA
2 6
Division of Clinical Oral Health Sciences, School of Dentistry, Division of Restorative Dentistry, International Medical University, Kuala
International Medical University, Kuala Lumpur, Malaysia Lumpur, Malaysia

© The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use,
sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and
the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this
article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included
in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will
need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The
Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available
in this article, unless otherwise stated in a credit line to the data.
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 2 of 13

Introduction Reviewing the literature on simulation exercises in

The increase in population, resource scarcity, and esca- disaster management & emergency preparedness is a
lating conflicts has increased both human-made and nat- complex task due to inconsistent use of labels and key-
ural disasters around the world. A medical response team words in the literature. [10]. Using PubMed, a database of
that is efficient and effective is crucial to improving the 36 journals has been created for emergency medicine, 24
outcome of disaster response. It is becoming increasingly for preparedness, and 24 for disaster management. Sev-
apparent that disaster medicine is progressing both theo- eral keywords are interchangeable, and many keywords
retically and practically over the years as more research is are used in other fields of medicine, resulting in false-
conducted [1]. There has been an increase in research on positive results. Finding the right titles and articles on
disaster preparedness in public health [2]. this subject is like searching for a needle in a haystack.
World Health Organization (WHO) has stated that Governments, non-governmental organizations, research
emergency relief supplies, preparedness, and prevention institutes, and universities published peer-reviewed and
measures are equally important [3]. In a situation like gray literature articles relating to disaster management.
this, disaster management and emergency preparedness
Further, despite the extensive use of SimEx, there
are of the utmost importance. However, current disaster
has been no systematic study of its efficacy and best
management training and education consist mainly of
practices. SimEx’s current disaster preparedness sta-
lectures and hospital drills. [4]. It is important to enhance
tus would provide valuable insight into its strengths
disaster readiness through health readiness and drills,
and limitations, as well as its potential for improve-
which prevents psychological, economic, physical, moral,
ment. Furthermore, this review may reveal gaps in
and financial harm to society. There needs to be a uni-
current knowledge and suggest future research direc-
form way of educating emergency medicine residents and
tions. A study might, for example, investigate vari-
other health professionals about disaster management.
ous forms of SimEx and assess the impact of differ-
Pandemics are worldwide health emergencies that take
ent aspects, such as stakeholder engagement and
some countries off guard, exposing serious flaws in their
technology use.
ability to deal with similar calamities. As a result of the
recent COVID-19 pandemic, the worlds mass emergency
response protocol has once again been debated at the In this study, we evaluated the current state of SimEx,
federal, state, and local levels. COVID-19 has highlighted the evaluation approaches of various simulation meth-
inadequacies in emergency planning among govern- ods, and the challenges that organizations face. In this
ments and international partners. Simulation Exercises work, we also examined the various types of SimEx in
(SimEx) can help prepare for future pandemics and other disaster management internationally and determine best
calamities by improving readiness and response skills [5]. practices. By systematically reviewing the existing litera-
Ultimately, the state of preparedness for an emergency on ture on SimEx in disaster management, the review aims
a local, state, and national level depends on first respond- to identify the different SimEx types that are being used
ers. During an emergency, healthcare professionals need internationally. Furthermore, the review aims to iden-
to be properly trained and equipped [6]. Human resource tify best practices for SimEx in disaster management as
development (HRD) entails a variety of areas for practi- well as explore challenges in conducting SimEx exercises,
tioners, such as relevant preparation, opportunities to which could ultimately improve their effectiveness.
practice new skills, drills, and exercises, and an assess-
ment of regional capabilities. Research question and objectives
More than half of the world’s population experienced The initial analysis question posed was “Which simula-
a disaster between 2005 and 2015, with increasing sever- tion exercises are effective and feasible in disaster pre-
ity and casualties [7]. The use of SimEx has become one paredness?”. Later it developed as “What are the current
of the most popular methods of training health care and state of disaster preparation SimEx and their effectiveness
allied professionals to manage disasters and emergencies in improving disaster response?“. This research question
in recent years [8]. It provides a safe and familiar envi- seeks to investigate the current state of SimEx practices
ronment for learning, allowing repetition to facilitate in disaster preparation, including their types, usage fre-
learning. SimEx provides healthcare professionals with quency, and effectiveness. The review would seek to
the opportunity to test their skills, develop effective strat- answer questions such as, What the key features of effec-
egies, and receive immediate feedback. SimEx enables tive SimEx are? What types of SimEx are most com-
disaster management teams to improve their decision- monly used? How frequently is SimEx conducted? and
making capabilities in a friendly, cooperative atmosphere what impact they have on disaster response? By answer-
[9]. ing these questions, the review will provide insights into
best practices for SimEx in disaster preparation and help
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 3 of 13

Table 1 Number of search strings with given keywords Table 2 Inclusion and exclusion criteria
Keywords Number of entries found Inclusion Criteria Exclusion Criteria
Disaster in emergency medicine 13,641 • Articles published after 2014 • Articles published
Catastrophe in disaster medicine 378 • English Language articles included after 2014
Exercise in disaster medicine 535 • Original studies, review articles, case reports, • Articles not in the
case series English language
Emergency exercise in disaster medicine 377
• Only articles published in peer-reviewed and • Editorials, opinions,
Simulation in disaster medicine 656 indexed journals correspondences
Drill in disaster medicine 224 • Non peer-reviewed/
Emergency in simulation exercises 955 non-indexed journals

to improve their effectiveness in reducing the disaster Several reviewers independently screened papers and
impact. assessed their quality and relevance to the research issue
The primary objective of the project was to under- during the review process. The final selection of papers
stand simulation exercises at present. The purpose of this for the review was based on reviewer consensus and con-
was to evaluate various types of simulation exercises in formity to the inclusion and exclusion criteria.
terms of effectiveness and feasibility. A detailed discus-
sion between the researchers helped determine second- Inclusion and exclusion criteria
ary objectives including how far SimEx can work as an Inclusion criteria included articles published in English
important educational tool for disaster preparedness to with at least one keyword corresponding to our reviewed
provide the desired outcome of enhanced field perfor- keywords in the title or abstract. The articles included
mance as well as obstacles and patterns in SimEx use. in this study have all been peer-reviewed and published
The rest of the article was framed as follows. Section 2 in high-quality journals. Articles published before 2014,
provides search strategies for article inclusion and crite- as well as articles in languages other than English, were
ria and quality assessment. Section 3 covers the adopted excluded from the study. Even though WHO guidelines
search results and an overview of study characteristics. were highlighted in documents before this period, they
In Sect. 4, SimEx’s findings and future directions are dis- were regularly updated by the organization. Non-peer-
cussed. Finally, Sect. 5 provides study conclusions and reviewed articles were also excluded from the final search
the scope of the present research. (Table 2). The process of searching and selecting studies
was organized according to the SPIDER question for-
Methods mat, which is a variation of the PICO tool. This process is
Document search illustrated in Fig. 1.
Document search was conducted using available lit-
erature extracted from the databases of PubMed (Med- Quality and risk bias assessment
line), Cumulative Index to Nursing and Allied Health Risk of bias assessment is a critical component for con-
Literature (CINAHL), Google Scholar, and Biomed cen- ducting any form of scientific review. The current study
tral by applying the Medical Subject Headings (MeSH). used Newcastle-Ottawa scale (NOS) (Table 3) to deter-
The search keywords ‘disaster in emergency medicine, mine the quality of the studies and bias risk assessment
‘catastrophe in disaster medicine’, ‘exercise in disaster [12]. The Newcastle-Ottawa scale is a quality assessment
medicine’, ‘emergency in exercise in disaster medicine’, tool that ranks the studies under review by designating
‘simulation in disaster medicine’, ‘drill in disaster medi- stars. The higher the number of stars* is an indication
cine, and ‘emergency in simulation exercises’ were used. of higher quality and less bias and a smaller number of
An increase in demand for SimEx in disaster manage- stars indicates the contrary. We used a modified version
ment prompted the need for this analysis as well as the of the NOS scale for this study which employed a 10-star
need for insight into this subject. According to the Pre- rating system instead of a commonplace, nine-star scale.
ferred Reporting Items for Systematic Reviews and The stars measure the quality of the studies in question
Meta-Analysis (PRISMA), guidelines specific open- on key fundamental aspects i.e. (selection, comparability,
ended questions were developed [11]. Table 1 presents and outcome). Interpretation of the NOS scale is fairly
the number of entries associated with each search key- simple, the studies are rated as poor (0–4*), fair (5–6*), &
word. The authors employed several keywords linked good ( 7–10*).
to disaster medicine and simulation exercises to search
multiple databases and established inclusion and exclu- Results
sion criteria, such as language (English), publication date Search outcomes
(after 2014-till date), and relevance to the research issue, A rigorous review of the titles and abstracts of each arti-
to choose which publications to include in their review. cle was conducted by the authors to ensure that they met
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 4 of 13

Fig. 1 Spider strategy of study selection

Table 3 Newcastle-Ottawa scale Quality assessment form for studies searched were included in a reference list for cata-
Non-Randomized Studies included in the review
loging any relevant articles that may have been missed by
Study 1 2 3 4 5 67 8 9 10 Score
*
the researchers at the beginning of the search. The litera-
Alim et al. 2015 * ** * * * 6
ture search identified 16,766 papers that are associated
Claudius et al. 2015 * ** ** * * * 8 with the given search terms. Since many search terms are
Zapko et al. 2015 * ** * ** * 7 used interchangeably, 6,572 duplicate works were found.
Arai et al. 2017 * ** * ** * * 8 Additionally, 248 records are discarded due to ineligibil-
Fogel et al. 2018 * ** * * ** * * * 10 ity because of no simulation component and focused on
Salway et al. 2018 * ** * ** * * * 9 disaster response rather than preparation, and 175 due
Mbanjumucyo et al. 2018 * * * ** * * 7 to other reasons such as lack of enough information on
Djalali et al. 2014 * ** * ** * * * 9 relevance or quality, and duplication. There were 9,801
Cramer et al. 2014 ** * ** * * * 8 articles reviewed for relevance and objectivity by the
Schulz et al. 2014 * ** * ** * * 8 authors. Following the application of inclusion and exclu-
ALuisio et al. 2016 * ** ** * * * 8 sion criteria, the authors excluded 9,226 articles. Quality
Johnson et al. 2017 * * * * * 5 check criteria were applied to the remaining 575 works
Hanson et al. 2018 * ** * ** * * * 9 using the NOS scale and quality scores were indepen-
Bentley et al. 2019 ** * ** * * 7 dently calculated using spreadsheets. Several discussions
Gomes et al. 2014 * * ** * * 6 followed, and 29 papers were selected for final review by
Kilianski et al. 2014 * ** * ** * * * 9 all authors. Figure 2 presents the PRISMA flowchart for
Shah et al. 2013 * ** * * ** * * 9 study selection.
Gist et al. 2016 * ** * * * 6
Lee et al. 2016 * ** * ** * * 8 Study characteristics
Obaid et al. 2017 ** * * ** * * 8 Many studies focused on SimEx to evaluate its opera-
Sheikhbardsiri et al. 2018 ** * ** * * 7 tional capabilities, system optimization, and staff for any
Ozella et al. 2019 ** * * ** * * * 9 emergency, including natural and man-made disasters.
Rüter A et al. 2016 * ** * * * * * 8 [13–20]. Table 4 presents the study characteristics based
Burke et al. 2014 * ** * ** * * 8 on SimEx type, design models, and guidelines.
James Le et al. 2020 * ** * ** * * 8
Foo Np et al. 2021 ** * ** * * 7
Exercise types
White Lewis et al. 2021 * ** * ** * * 8
Drill exercises were largely highlighted in studies [13, 21–
Nejadshafiee et al. 2022 * ** * * * 6
24] followed by full-scale exercises [14, 17, 20, 24–29] and
Chen et al. 2019 * ** * * * * 7
tabletop exercises [16, 18, 19]. The other type of studies
reviewed included workshop discussion-based simula-
the inclusion and exclusion criteria. Those abstracts that tion exercises [13, 25, 30, 31], computer-based simulation
lacked sufficient data were carefully assessed for signifi- exercise [32], and operational-based functional exercises
cance and relevance and the search strategy was system- [33–36]. The studies used different evaluation and assess-
atic based on a rational and step-by-step process. All the ment mechanisms to assess the impact and success of
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 5 of 13

Fig. 2 Study selection flow-chart based on PRISMA 2021 guidelines

this simulation exercise. The studies used different evalu- and METHANE which is a standard tool to assess and
ation and assessment mechanisms to assess SimEx’s suc- report major incident parameters include “type”, “pre-
cess and impact. cise location”, “hazards”, “access”, “casualty numbers”, and
“emergency service [28]. To evaluate them, it is impera-
SimEx Design models tive to have a clear and concise strategy. All stakehold-
To assess SimEx quality and effectiveness, authors need ers should be able to assess the effectiveness of these
to clarify the techniques and evaluation types used. It’s exercises based on scientific theory and supported by
because SimEx exercises and disaster drills consume research. Performing these evaluations can make disas-
an enormous amount of time and money. The follow- ter SimEx’s financially viable and attract funding for
ing evaluation methods were most commonly used in research.
the studies: Questionnaires/Likert scales [18, 23, 25, 29, An evaluation rationale should be grounded in empiri-
30, 32, 36–38] debriefing pre sessions [17, 21] and post- cal evidence, and the authors should explain which
exercise exam [16, 37, 39]. The other evaluation methods methods they will use to achieve their goals. The steps
used to assess the SimEx’s outcomes were observed for described above will lead to a logical and scientific
live scenarios and learning management system software approach to evaluating disaster SimEx and building inter-
such as Moodle [20], retrospective object evaluation [22], est in this area. According to the hospital safety index
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 6 of 13

Table 4 Study characteristics

N Study type Country Disciplinary Participants SimEx Type Design Guidelines Ref
1 Original Indonesia Single 309 students Discussion Pre-test and post-test International Council (Alim et
research in the training and for in-class training, of nurses’ framework al.)
program, operational observation for disas- of disaster nursing
225 in a disaster exercises ter drill, and in-depthcompetency, cross-
drill interview cutting competencies
for healthcare workers in
disaster training, and core
competencies for nurses
in emergency and disaster
preparedness.
2 Original USA Single 33 students Operational Observation of live Simple triage and rapid (Claudi-
research (four live pa- exercises (drill) scenarios and learning treatment (JumpSTART) us et
tients and seven management mass casualty triage tool al.)
computerized system software
patients) (Moodle systems;
Perth, Australia)
for computerized
scenarios
3 Pilot study USA Multi 132 nursing Operational Debriefing session Jeffries simulation (Zapko
students exercises (drill) (According to the framework et al.)
and 25 radiology Agency for Healthcare
students Research and Quality
AHRQ, 2011)
4 Original Tokyo, Single 103 Operational Objective evaluation A Manual was created by (Arai et
research Japan (Players:58 exercises (drill) (retrospective) Tokyo universities with al.)
Simulators:22 a scheduled time to
Instructors:13 complete tasks, necessary
Others:10) staff members, detailed
supplies and equip-
ment lists, and formatted
documents/forms.
5 Observational Israel Multi 178 healthcare Operational The self-administered The platform was the (Fogel
workers exercises (Bio- questionnaire was Israeli “Orange Flame” et al.)
terrorism drill) evaluated under ob- exercise,
servation according to a national preparedness
a scoring technique of buildup project conduct-
the skills competency ed by the Israeli Ministry
checklist for contact of Health per Centers for
precautions with the Disease Control and Pre-
CDC vention (CDC) guidelines
6 Descriptive New York Multi 62 participants, Vertical evacu- Self-evaluation form Evacuation protocol was (Salway
survey 13 healthy vol- ation drill. of 15 questions based developed to review the et al.)
unteer patients (full-scale on a Likert scale of hospital evacuation plans
exercise) one to five (Poor to across the USA and to
Excellent) modify existing plans and
protocols based on the
institution’s old evacua-
tion drills.
7 Case report Rwanda Multi 174 volunteers Karongi (If METHANE is a Standard color-coded (Mban-
(Africa) the boat standardized method visual assessment triage: ju-
capsized), and to assess and report red, yellow, and green in mucyo
Kanombe major incidents major incident medical et al.)
(If the plane management and support
crashed) (MIMMS) course
exercises
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 7 of 13

Table 4 (continued)
N Study type Country Disciplinary Participants SimEx Type Design Guidelines Ref
8 Original Piedmont Single 61 casualties Full-scale hos- Level of disaster pre- Simple triage and rapid (Djalali
research region of pital exercise paredness of hospital transport (START) triage et al.)
Italy (Explosion of a staff based on HIS
gas station) guidelines and re-
sponse performance
evaluation based
on the CRIMEDIM
Method
9 Report Tunisia Multi 31 participants, Field-based Facilitators from WHO The core humanitarian (Cran-
(19 from WHO- exercise and HTI competencies framework mer et
related missions used the competen- developed by the consor- al.)
and 12 from cy-based evaluation tium of British humanitar-
nongovernmen- tool designed for ian agencies (CBHA)
tal humanitarian this SimEx, evidence-
training initiative based evaluation,
(HTA) agencies. Follow-up interview
10 Prehospital Germany Multi 75 human Prehospital HA and every HFS Triage classifications (Schulz
and Disaster actors, 4 four mass causality technician recorded according to the German et al.)
Medicine, high-fidelity incident (MCI) important time points association of emergency
Report simulators drill and type and number physicians
of diagnostic and
therapeutic tasks on
a paper and pencil
questionnaire and
reviewed videotapes
and checklist scoring
11 Random Con- Lucknow, Single 60 Nurses SimEx, and Two standardized American board of emer- (Aluisio
trol Trail India discussion- multiple-choice gency medicine model et al.)
based question batteries, en- of the clinical practice of
exercises compassing key core emergency medicine with
content were used for the topical focus of triage
assessments in disaster situations. Addi-
tional content was drawn
from the CDC guidelines
and standard emergency
medicine reference texts
12 Survey Sweden Single 13 Nurses Small-scale Pre and post-test Quantitative experimental (Jonson
computer- questionnaires method within-group et al.)
based design, prototype training
simulation system called Dig Emergo
exercise
13 Pilot Latin Multi Ministry of Workshop, A combination of the GRPT and internation- (Han-
America health repre- table-top Kirkpatrick model of alization process of the son et
sentatives from (Discussion- training evaluations, Homeland security and al.)
4 countries in based pre and post-course Exercise Evaluation Pro-
Latin America exercise) exams, targeted ac- gram (HSEEP)
tivities, and a delayed
participant survey
enabled the Global
response prepared-
ness team (GRPT) to
address 3 of the 4
levels of evaluation.
14 Original USA Multi 174 Operational During exercise: Multi patient (Bentley
research exercise checklist MCI triage Sim Wars et al.)
Post-exercise:
Response options on
a 4-point Likert-like
scale
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 8 of 13

Table 4 (continued)
N Study type Country Disciplinary Participants SimEx Type Design Guidelines Ref
15 Case study Brazil Multi 26 different Full-scale CTA Techniques, direct External Emergency Plan (Gomes
agencies exercise observations, and (EEP) et al.)
electronic records of
audio/video
16 Original Illinois Multi CCDPH staff, Full-scale Debriefing for imme- CCDPH has adopted the (Kilian-
research local and exercise in diate evaluation, Federal Emergency Man- ski et
regional volun- response to Observation agement Agency (FEMA) al.)
teers, and the a simu- National preparedness
local municipal lated anthrax cycle. Using FEMA’s HSEEP
police and fire bioterrorism principles.
departments attack
17 Report USA Multi 36 actors/pa- Full-scale retrospectively Kings County Hospital (Shah
tients (medical chlorine evaluated electronic Center’s (KCHC’s) PICU (Pe- et al.)
students or overexposure medical record diatric Intensive Care Unit)
emergency exercise surge plan,
medicine which was developed
residents) in conjunction with the
Pediatric Disaster
The coalition is a member
of the New York city
department of Health.
18 Original USA Multi Emergency Full-scale Questionnaire “Disaster Olympics” study (Gist et
research medicine exercise design al.)
residents,
Medical Reserve
Corps (MRC)
volunteers
19 Report Thailand Multi 66 personnel 3-day training, Videotaped, survey Exercise co-hosted by (Lee et
from the Korea table questionnaires, Korea and Thailand, al.)
Disaster Relief top exercises interview Third ARF DiRex
Team, 40 medi- (ASEAN Regional Forum
cal professionals, Disaster Relief Exercise)
and 106 military
personnel.
20 Report USA Multi 667 participants Six functional Post-exercise HSEEP exercise planning (Obaid
and 83 com- exercises HSEEP participant guidelines were used for et al.)
mand structures, feedback forms, exercise development by
three Medical IC-specific exercise the CPE exercise design
Response Sys- evaluation guide, team, Incident Command
tems (MRS) hot wash feedback, (IC) system framework
observation
21 Report Southeast Multi 990 Volunteers 2-day func- checklist consisted of The crisis management or- (Sheikh-
Iran. tional exercise 13 functional dimen- ganization of the ministry bardsiri
sions based on the of interior and the accredi- et al.)
Iranian emergency tation office of the Iranian
operation plan (EOP) ministry of health have
provided guidelines and
instructions for disaster
22 Original Italy Multi 238 participants Functional Observation, The framework of the (Ozella
research exercise wearable proximity residential course of the et al.)
sensors European Master in Disas-
ter Medicine (EMDM)
23 Report Sweden Multi Staff and Two tabletop The Hospital Incident The Emergo Train model (Rüter
managers exercises Command System (ETS) was used as the et al.)
of two local (HICS) and the simulation tool
hospitals Disaster Management
Indicator model (DiMI)
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 9 of 13

Table 4 (continued)
N Study type Country Disciplinary Participants SimEx Type Design Guidelines Ref
24 Original USA Multi In three local Full-functional Observation, Mixed methods to com- (Burke
research hospitals, staff disaster Interviews, prehensively assess the et al.)
including physi- exercise quantitative and current state of disaster
cians, nurses, qualitative feedback preparedness, evidence-
and nonclinical from exercise partici- based, pediatric-specific
workers pants and observers disaster triage systems
25 Randomized Haiti Multi 480 community Three days of 3 interviews were Community-based, inte- (James
Control Trail members educational conducted after the grated disaster prepared- et al.)
and training intervention of base- ness randomized control
exercise line, 3 and 7 months trial
post-intervention
respectively
26 Observational Taiwan Multi This full scale Full-Scale The researchers as- Full-scale exercise orga- (Foo et
was carried out exercise signed 6 scholars from nized by A Nan Hospital of al.)
jointly between Taiwan’s society of China Medical University
8 DMAT teams emergency medicine and the fire bureau of
and 86 USAR and a non-govern- Tainan City in collabora-
teams ment expert from tion with the city govern-
Hong Kong to work ment and the Ministry of
as exercise evaluators Health
and examiners. every
expert was respon-
sible for assessments
at different times and
in different disaster
scene areas.
27 Clinical trial USA Single 31 students Educational Used a pre and post- The experimental group (White-
and training intervention survey was given training on Lewis et
exercise in to determine the bioterrorism, the START al.)
disaster pre- impact of a disaster method, demonstration
paredness for preparedness educa- and return demonstration
high school tion intervention. The of first aid, then shock,
students. tool utilized was the and bleeding treatments.
adapted Emergency Practical training with
Preparedness Informa- spine board and cervical
tion Questionnaire collar application
(EPIQ)
28 Cross-sectional Iran Multi 21 Specialists A functional The nurses present The operations-based (Nejad-
exercise (drill) at the exercise site exercise scenarios that shafiee
for the worked in teach- included the scale of et al.)
possibil- ing hospitals in the earthquake, victims’
ity of nursing Kerman and sent the number, the affected area,
interventions information related to injury type, and the local
hypothetical injuries hospital capacity was
to experienced nurses done in Kerman Medical
using the equipment hospital.
(Internet) available at
the exercise site and
provided the target
care after receiving
their responses.
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 10 of 1
3

Table 4 (continued)
N Study type Country Disciplinary Participants SimEx Type Design Guidelines Ref
29 Observational Taiwan Multi 40 standardized Three identi- Written in the script As part of the SimEx (Chen
patients cally designed were all of the training, participants et al.)
full-scale simulated injuries set up temporary field
exercises that needed to be medical stations while
performed on the classifying and treating
recruited patients. simulated patients. Triage
To test participants’ and treatment areas
performance, a mock were separate at the field
wound makeup medical stations. Accord-
based on their scripts ing to the script, they
was applied to the performed their injuries
simulated wound. and reacted to manage-
ment instructions.

(HSI) guidelines level of preparedness of hospital staff in [41]. It is also claimed that when disaster exercise assess-
disasters is necessary to control the severity [14]. Com- ments are based on both quantitative and qualitative
petency-based evaluation tools designed for this SimEx data, the evaluation conclusions are more meaningful
included follow-up interviews (Cranmer et al.) and cog- [42]. Disaster exercises offer certain advantages in terms
nitive task analysis (CTA) [27], Post-exercise HSEEP of convenience of use, function-driven nature, precision,
participant feedback forms, and IC-specific exercise consistency, validity, reliability, and cultural consider-
evaluation guide [35]. One assessment tool was used in ations [43, 44]. Therefore, the primary study objective of
some studies, while others were examined using a variety identifying and investigating patterns was achieved by a
of tools including observation, interviews, proximity sen- better understanding of the SimEx models that are cur-
sors, hot wash feedback, and electronic records [20, 26, rently being used in various disaster planning and emer-
33, 35]. gency response scenarios.
In the absence of research, it is not known whether
Guidelines development for SimEx exercise assessment techniques are effective or superior.
SimExs are developed systematically and scientifically However, one evaluation method, such as video and pho-
using an exercise manual and simulation guidelines. tography, has been evaluated and found to be efficient
When creating scenario-based SimExs, exercise organiz- [14, 18, 27, 38]. Video evaluations can provide some
ers must adhere to certain guidelines for scenario devel- benefits, such as the ability to evaluate better, provide a
opment, delivery, and after-action reviews. Planning the secondary evaluation, and display the participant’s per-
exercise should begin with a description of the research formance. The time saved by video debriefing can also
question or learning objectives that will be addressed be used for learner rehabilitation, scenario modifica-
[13, 22]. The development of SimEx should also be car- tion, or other instructional tasks. According to studies,
ried out by professionals who examine previous research, numerous tools are available for assessing disaster recov-
plans, systems, and simulation designs. It is also impor- ery exercises. The inclusion of such tools is merely the
tant to develop a risk assessment plan to address the risks responsibility of hospitals and is frequently dispensed as
related to the SimEx design and participants [28]. Diverse function-based in checklists. There is no comprehensive
groups of participants with defined roles assigned via dif- tool that can apply to all healthcare systems, including
ferent teams make exercises more dynamic and enjoy- well-being, therapeutic interventions, and assistance [14,
able [40]. Invitations should be based on a stakeholder 17, 19, 20, 38].
analysis to determine which participants would be most In the present analysis, we included large-scale stud-
appropriate for the exercise’s learning objectives [15, 37]. ies to understand the potentiality of SimEx approaches in
disaster preparedness [14, 17, 20, 24, 26]. It is reported
Discussion that choosing evaluators was an essential part of SimEx
The review analyzed the diverse range of SimEx exer- designing since they are potentially influenced by the
cises carried out in disaster and emergency medicine and evaluator’s perception, judgment, and knowledge of
compared the SimEx practices observed in various stud- disaster management, critical care, and preparedness
ies. The study also examined the challenges & obstacles [17, 18, 20, 33]. Our analysis shows that the majority
to effective SimEx implementation and proposed spe- of SimEx exercises conducted around the world were
cific recommendations to enhance disaster prepared- tabletop exercises. Studies suggest that discussion-based
ness and response plans, processes, and systems, based tabletop exercises are the easiest to organize, conduct,
on the 2017 report by the World Health Organization and evaluate, especially when there are a large number of
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 11 of 1
3

participants. Exercises based on scenarios (such as drills In the future, researchers and practitioners can use the
or full-scale exercises) require more preparation, financ- current study results on SimEx in disaster preparation to
ing, and organization [45]. improve its effectiveness in a variety of ways, including.
Simulators, drills, and training sessions are increas- •  Creating standardized SimEx protocols: Research
ingly being integrated into post-graduate medical and practitioners can use the study findings to
training courses around the world. There has been a create standardized SimEx protocols, which can
massive transformation in the medical world in terms of ensure consistency in SimEx practices and facilitate
resources, infrastructure, technology, and public research comparison between programs.
in recent decades. Employees have been equipping them- •  Integrating more rigorous assessment: SimEx
selves with the necessary skills to deal with catastrophes programs may benefit from a more rigorous
as part of this transformation. This has led to the creation evaluation, which can identify areas for improvement
of disaster simulation centers and the conduct of SimEx and ensure that they are meeting their intended
at universities and colleges. Super-specialized organiza- goals.
tions have also emerged [46]. Further, studies by Luan D •  Addressing implementation barriers: SimEx
et al., Huang S et al. and Lyu et al. highlight the impor- implementation may be hampered by resource
tance of incorporating natural hazard risk assessment constraints or stakeholder apathy. It may be
and emergency response planning into infrastructure possible to overcome these hurdles in the future by
design and planning. They also showcase the potential developing solutions to solve resource constraints or
benefits of using advanced modeling and optimization by including stakeholders early on in SimEx design.
techniques to improve the performance and resilience of •  Finding opportunities for improvement: The
infrastructure systems. Overall, these articles contribute assessment may indicate particular areas for
to a growing body of research aimed at promoting the improvement in SimEx procedures, such as the need
safety, sustainability, and resilience of infrastructure sys- for more realistic scenarios or greater interaction
tems in the face of natural hazards and other challenges with other disaster management systems. Future
[46–48]. studies or practices can concentrate on tackling these
The health sector is lacking evidence and information specific areas to increase SimEx’s efficacy.
regarding the interminable implications of exercise on Overall, the findings of a systematic assessment of disas-
preparedness and response in an emergency. Participat- ter simulation exercises may be utilized to drive the
ing in SimEx results in improved emergency plans that development and implementation of SimEx techniques,
will lead to an understanding of the weaknesses and resulting in better disaster preparedness and response.
limitations of an individual or an organization. This does
not guarantee that this understanding leads to actual Conclusion
improvement and more effective emergency manage- In this study, we examined current methodologies for
ment. It is difficult to demonstrate SimEx’s effectiveness evaluating safety interventions following accidents and
at an institutional level as long as there is no evidence of disasters. The healthcare industry has conducted drills
any long-term positive effect on public health emergency and operations to prepare for disasters and accidents.
preparedness. Mass casualty incidents (MCI) are characterized as
Globally, there has been a lack of consistency in the overpowering events in which patients outweigh locally
response to the COVID-19 pandemic. Various countries available resources. These events require a robust emer-
have taken sequestered responses to worldwide prob- gency response which usually necessitates support from
lems, which makes greater preparation for pandemics, the state or region. [49].Diverse approaches and meth-
disasters, and simulations imperative. Moreover, more ods should be used according to the type and purpose
research is required in the area of disaster preparedness of the activity. Healthcare facilities may use a variety
to complete the knowledge gap. There is no evidence to of approaches and strategies to plan safety actions and
support the effectiveness of current exercise assessment assess disaster response. Exercises can be tabletop, func-
techniques. To determine the usefulness of various forms tional, or full-scale and are used to evaluate emergency
of exercise assessment techniques in the future, emer- response protocols and highlight areas for improve-
gency management experts should conduct immersive ment. Performing post-incident evaluations to assess
experiments. Using the current available evaluation tools response activities’ efficacy and suggest improvements.
and strategies, this review will contribute to improving Training employees to be prepared to handle crises by
the readiness of various sectors of the healthcare system. creating and executing training programs. Establishing
In this way, disaster management can be implemented alliances and collaborations with other organizations to
successfully. share resources and improve response skills. It is recom-
mended that disaster preparedness in healthcare requires
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 12 of 1
3

emergency medicine residency programs in the United States. Prehosp

a multifaceted approach. In this regard, it is essential to Disaster Med. 2017;32(4):368–73.
consider the specific needs, resources, and goals of the 5. Biswas S, Bahouth H, Solomonov E, Waksman I, Halberthal M, Bala M. Pre-
organization. paredness for mass casualty incidents: the effectiveness of current training
model. Disaster Med Pub Health Prep. 2022;16(5):2120–8.
Abbreviations 6. Choi HS, Lee J-E. Hospital nurses’ willingness to respond in a disaster. JONA:
CINAHL Cumulative Index to Nursing and Allied Health Literature The Journal of Nursing Administration. 2021;51(2):81–8.
CTA Cognitive task analysis 7. Norris FH, Stevens SP, Pfefferbaum B, Wyche KF, Pfefferbaum RL. Community
HRD Human resource development resilience as a metaphor, theory, set of capacities, and strategy for disaster
HSI Hospital safety index readiness. Am J Community Psychol. 2008;41:127–50.
MCI Mass casualty incidents 8. Järvensivu P, Räisänen H, Hukkinen JI. A simulation exercise for incorporat-
MeSH Medical Subject Headings ing long-term path dependencies in urgent decision-making. Futures.
NOS Newcastle-Ottawa Scale 2021;132:102812.
PRISMA Preferred Reporting Items for Systematic Reviews and 9. Guo Y, Yang H, Chen M, Cheng J, Gong D. Ensemble prediction-based
Meta-Analyses dynamic robust multi-objective optimization methods. Swarm Evol Comput.
SimEx Simulation exercise 2019;48:156–71.
WHO World Health Organization 10. Birnbaum ML, Adibhatla S, Dudek O, Ramsel-Miller J. Categorization and
analysis of disaster health publications: an inventory. Prehosp Disaster Med.
Acknowledgements 2017;32(5):473–82.
None. 11. Page M, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C, Moher D.
The PRISMA 2020 statement: an updated guideline for reporting systematic
Authors’ contributions reviews. Syst reviews. 2021;10(1):1–11.
Design and Data Collection: Syed Sarosh Mahdi (SSM), Raheel Allana (RA), Gopi 12. Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M et al. The
Battineni (GB)Final Manuscript Revision:Kiran Rehman(KR) ,Syed Sarosh Mahdi Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised
(SSM) & Francesco Amenta (FA)Manuscript Preparation: Syed Sarosh Mahdi studies in meta-analyses. 2013. 2016.
SSM), Raheel Allana (RA), Gopi Battineni (GB). Sakina Syeda (SS)Methods: Syed 13. Alim S, Kawabata M, Nakazawa M. Evaluation of disaster prepared-
Sarosh Mahdi (SSM), Raheel Allana (RA), Gopi Battineni (GB), Kiran Rehman ness training and disaster drill for nursing students. Nurse Educ Today.
(KR)Paper Reviews: Syed Sarosh Mahdi, Raheel Allana, Gopi Battineni, Sakina 2015;35(1):25–31.
Syeda & Mariam Khawaja Research Background: Syed Sarosh Mahdi, Francesco 14. Djalali A, Carenzo L, Ragazzoni L, Azzaretto M, Petrino R, Della Corte F, et al.
Amenta, Mariam Khawaja, Daniyal AghaStudy Approval: Syed Sarosh Mahdi, Does hospital disaster preparedness predict response performance during a
Sakina Syed, Mariam Khawaja (MK)Study Planning: Syed Sarosh Mahdi, Raheel full-scale exercise? A pilot study. Prehosp Disaster Med. 2014;29(5):441–7.
Allana, Gopi Battineni,Francesco AmentaStudy Selection: Kiran Rehman, 15. Cranmer H, Chan JL, Kayden S, Musani A, Gasquet PE, Walker P, et al. Devel-
Mariam Khawaja, Daniyal Agha, Sakina SyedaAll authors have read and agreed opment of an evaluation framework suitable for assessing humanitarian
to the published version of the manuscript. workforce competencies during crisis simulation exercises. Prehosp Disaster
Med. 2014;29(1):69–74.
Funding 16. Hanson K, Hernandez L, Banaski JA Jr. Building simulation exercise capac-
The research received no funding from any funding agency. ity in Latin America to manage public health emergencies. Health Secur.
2018;16(S1):–98.
Data Availability 17. Kilianski A, O’Rourke AT, Carlson CL, Parikh SM, Shipman-Amuwo F. The plan-
The datasets used and/or analyzed during the current study are available from ning, execution, and evaluation of a mass prophylaxis full-scale exercise in
the corresponding author upon reasonable request. cook county, IL. Biosecurity and bioterrorism: biodefense strategy. Pract Sci.
2014;12(2):106–16.
18. Lee JI, Lee KH, Kim OH, Cha YS, Hwang SO, Kim H, et al. Evaluation of an inter-
Declarations national disaster relief team after participation in an ASEAN regional forum
disaster relief exercise. Disaster Med Pub Health Prep. 2016;10(5):734–8.
Competing interests 19. Rüter A, Kurland L, Gryth D, Murphy J, Rådestad M, Djalali A. Evaluation of
The authors declare no competing interests. disaster preparedness based on simulation exercises: a comparison of two
models. Disaster Med Pub Health Prep. 2016;10(4):544–8.
Ethics approval and consent to participate 20. Burke RV, Kim TY, Bachman SL, Iverson EI, Berg BM. Using mixed methods
Not applicable. to assess pediatric disaster preparedness in the hospital setting. Prehosp
Disaster Med. 2014;29(6):569–75.
Consent to publish 21. Zapko KA, Ferranto ML, Brady C, Corbisello A, Hill D, Mullen R, et al. Interdis-
Not applicable. ciplinary disaster drill simulation: laying the groundwork for further research.
Nurs Educ Perspect. 2015;36(6):379–82.
Received: 26 August 2022 / Accepted: 17 May 2023 22. Arai T, Ohta S, Onishi M, Taniguchi M, Tsurukiri J, Kumasaka K, et al. Empiri-
cal research on the utility of a preparation manual for a disaster medical
response drill. Open Access Emergency Medicine: OAEM. 2017;9:81.
23. Fogel I, David O, Balik CH, Eisenkraft A, Poles L, Shental O, et al. The associa-
tion between self-perceived proficiency of personal protective equipment
and objective performance: an observational study during a bioterrorism
simulation drill. Am J Infect Control. 2017;45(11):1238–42.
References 24. Foo N-P, So EC, Lu N-C, Hsieh S-W, Pan S-T, Chen Y-L et al. A 36-hour
1. Guha-Sapir D, Vos F, Below R, Ponserre S. Annual disaster statistical review unplugged full-scale exercise: closing the gaps in interagency collaboration
2011: the numbers and trends. 2012. between the disaster medical assistance team and urban search and rescue
2. Heidaranlu E, Ebadi A, Khankeh HR, Ardalan A. Hospital disaster preparedness team in disaster preparedness in taiwan. Emergency medicine international.
tools: a systematic review. PLoS Curr. 2015;7. 2021;2021.
3. Yeager VA, Menachemi N, McCormick LC, Ginter PM. The nature of the public 25. Gist R, Daniel P, Grock A, Lin C-j, Bryant C, Kohlhoff S, et al. Use of medical
health emergency preparedness literature 2000–2008: a quantitative analysis. reserve corps volunteers in a hospital-based disaster exercise. Prehosp Disas-
J Public Health Manage Pract. 2010;16(5):441–9. ter Med. 2016;31(3):259–62.
4. Sarin RR, Cattamanchi S, Alqahtani A, Aljohani M, Keim M, Ciottone GR. 26. Shah VS, Pierce LC, Roblin P, Walker S, Sergio MN, Arquilla B. Waterworks, a full-
Disaster education: a survey study to analyze disaster medicine training in scale chemical exposure exercise: interrogating pediatric critical care surge
Mahdi et al. BMC Emergency Medicine (2023) 23:52 Page 13 of 1
3

capacity in an inner-city tertiary care medical center. Prehosp Disaster Med. 39. Claudius I, Kaji A, Santillanes G, Cicero M, Donofrio JJ, Gausche-Hill M, et al.
2014;29(1):100–6. Comparison of computerized patients versus live moulaged actors for a
27. Gomes JO, Borges MR, Huber GJ, Carvalho PVR. Analysis of the resilience of mass-casualty drill. Prehosp Disaster Med. 2015;30(5):438–42.
team performance during a nuclear emergency response exercise. Appl 40. Nejadshafiee M, Nekoei-Moghadam M, Bahaadinbeigy K, Khankeh H,
Ergon. 2014;45(3):780–8. Sheikhbardsiri H. Providing telenursing care for victims: a simulated study
28. Mbanjumucyo G, Nahayo E, Polzin-Rosenberg N, Cattermole GN. Major for introducing of possibility nursing interventions in disasters. BMC Med Inf
incident simulation in Rwanda: a report of two exercises. Afr J Emerg Med. Decis Mak. 2022;22(1):54.
2018;8(2):75–8. 41. Organization WH. WHO simulation exercise manual: a practical guide and
29. Salway RJR, Adler Z, Williams T, Nwoke F, Roblin P, Arquilla B. The challenges of tool for planning, conducting and evaluating simulation exercises for
a vertical evacuation drill. Prehosp Disaster Med. 2019;34(1):25–9. outbreaks and public health emergency preparedness and response. World
30. James LE, Welton-Mitchell C, Noel JR, James AS. Integrating mental health Health Organization; 2017.
and disaster preparedness in intervention: a randomized controlled trial 42. Rajabi S, Khankeh HR, Delshad V, Rahgozar M, Arsalani N, Mohammadi F.
with earthquake and flood-affected communities in Haiti. Psychol Med. Effects of comprehensive risk management program on the preparedness of
2020;50(2):342–52. rofeide rehabilitation hospital in disasters and incidents. Health in Emergen-
31. White-Lewis S, Beach E, Zegers C. Improved knowledge of disaster prepared- cies and Disasters Quarterly. 2017;2(4):179–86.
ness in underrepresented secondary students: a quasi‐experimental study. J 43. Agboola F, McCarthy T, Biddinger PD. Impact of emergency preparedness
Sch Health. 2021;91(6):490–8. exercise on performance. J Public Health Manage Pract. 2013;19:77–S83.
32. Jonson C-O, Pettersson J, Rybing J, Nilsson H, Prytz E. Short simulation 44. Savoia E, Biddinger PD, Burstein J, Stoto MA. Inter-agency communication
exercises to improve emergency department nurses’ self-efficacy for initial and operations capabilities during a hospital functional exercise: reliability
disaster management: controlled before and after study. Nurse Educ Today. and validity of a measurement tool. Prehosp Disaster Med. 2010;25(1):52–8.
2017;55:20–5. 45. Skryabina E, Reedy G, Amlot R, Jaye P, Riley P. What is the value of health
33. Ozella L, Gauvin L, Carenzo L, Quaggiotto M, Ingrassia PL, Tizzoni M, et al. emergency preparedness exercises? A scoping review study. Int J disaster risk
Wearable proximity sensors for monitoring a mass casualty incident exercise: Reduct. 2017;21:274–83.
feasibility study. J Med Internet Res. 2019;21(4):e12251. 46. Luan D, Liu A, Wang X, Xie Y, Wu Z. Robust two-stage location allocation for
34. Sheikhbardsiri H, Yarmohammadian MH, Khankeh H, Khademipour G, Mora- emergency temporary blood supply in postdisaster. Discrete dynamics in
dian MJ, Rastegarfar B, et al. An operational exercise for disaster assessment nature and society. 2022;2022:1–20.
and emergency preparedness in south of Iran. J Public Health Manage Pract. 47. Huang S, Huang M, Lyu Y. Seismic performance analysis of a wind turbine
2020;26(5):451–6. with a monopile foundation affected by sea ice based on a simple numerical
35. Obaid JM, Bailey G, Wheeler H, Meyers L, Medcalf SJ, Hansen KF, et al. method. Eng Appl Comput fluid Mech. 2021;15(1):1113–33.
Utilization of functional exercises to build regional emergency prepared- 48. Huang S, Lyu Y, Sha H, Xiu L. Seismic performance assessment of unsaturated
ness among rural health organizations in the US. Prehosp Disaster Med. soil slope in different groundwater levels. Landslides. 2021;18(8):2813–33.
2017;32(2):224–30. 49. Lomaglio L, Ansaloni L, Catena F, Sartelli M, Coccolini F. Mass Casualty
36. Bentley S, Iavicoli L, Boehm L, Agriantonis G, Dilos B, LaMonica J, et al. A Incident: definitions and current reality. In: Kluger Y, Coccolini F, Catena F,
simulated mass casualty incident triage exercise: SimWars. MedEdPORTAL. Ansaloni L, editors. WSES Handbook of Mass casualties incidents manage-
2019;15:10823. ment. Hot topics in Acute Care surgery and trauma. Cham: Springer; 2020.
37. Aluisio AR, Daniel P, Grock A, Freedman J, Singh A, Papanagnou D, et al. Case-
based learning outperformed simulation exercises in disaster preparedness
education among nursing trainees in India: a randomized controlled trial. Publisher’s Note
Prehosp Disaster Med. 2016;31(5):516–23. Springer Nature remains neutral with regard to jurisdictional claims in
38. Schulz CM, Skrzypczak M, Raith S, Hinzmann D, Krautheim V, Heuser F, published maps and institutional affiliations.
et al. High-fidelity human patient simulators compared with human
actors in an unannounced mass-casualty exercise. Prehosp Disaster Med.
2014;29(2):176–82.