ROUTINE HEALTH INFORMATION SYSTEMS
A Curriculum on Basic Concepts and Practice
MODULE 5:
RHIS Data Analysis
SESSION 2:
Overview of Steps 1-4 of Data Analysis
The complete RHIS curriculum is available here:
https://www.measureevaluation.org/our-work/ routine-health-information-systems/rhis-
curriculum
1
Learning Objectives and Topics
Covered
Objectives
By the end of this session, participants will be able to:
• Select appropriate indicators for data analysis
• Conduct a basic desk review of data quality and adjust data if
necessary
• Select appropriate denominators
• Compare findings from routine data with findings from other data
sources
• Analyze routine data to produce information products (tables, graphs,
and maps)
Topics Covered
• Selection of indicators for analysis
• Desk review of data completeness and internal consistency
• Selection of appropriate denominators
• Comparison of findings from routine data with findings from other data 2
5 Steps of Data Analysis
1. Select a limited set of core indicators
2. Review data quality
3. Select appropriate denominators
4. Reconcile findings with estimates from
other data sources
5. Communicate key findings (to be
discussed in Module 5, Session 3)
3
S1: First Step
Select a Limited Set of Core Indicators
4
What Makes an Indicator “SMART”?
(Characteristics of Good Indicators)
• Specific: Indicator is concrete, detailed, focused,
and well-defined
• Measurable: Indicator tells how many or how much
and can be measured with identified measurement
sources
• Agreed upon: Stakeholders vested in a specific
M&E question should agree that indicator is relevant
• Relevant: Indicator generates data that can
answer the question of interest
• Time-bound: Indicator specifies time frame of what
it is measuring
5
S1: Select a Limited Set of Core
Indicators
Core indicators should:
• Be based on M&E framework for the national
health sector strategy
• Be programmatically relevant, and facilitate
program management
• Reliably and comprehensively assess the
performance of the health system (whether
national or subnational levels)
• Be clearly defined
• Have numerators that are measurable with
routine health data
6
S1: Core Indicator with 2 Data Sources: DTP3
Routine health Numerator: Number of infants
information systems immunized with DTP3 by 12
months of age in a given year
Denominator: Total number of
surviving infants <12 months of
age in same year
Population-based Numerator: Number of children
survey ages 12–23 months who received
three doses of DTP3 vaccine by
age 12 months
Denominator: Total number of
children ages 12–23 months
surveyed
7
S1: Core Indicators Measured Reliably
with One Data Source
Antiretroviral Numerator: Number of adults and
therapy (ART) children with HIV, alive and on
retention rate antiretroviral therapy (ART) 12, 24, 36
months (etc.) after initiating
Source: Routine health treatment Denominator: Total
information systems number of patients initiating ART
during a specific period
Fully-immunized Numerator: Number of children ages
child 12–23 months who received 3 doses
of OPV, 3 doses of DTP, and 1 dose
Source: Population- each of BCG and measles vaccine
based survey before age 12 months
Denominator: Total number of
children ages 12–23 months surveyed
8
Some Indicators Are Not Clearly Defined or
Cannot Be Reliably Measured with Routine
Data
Some indicators are not clearly defined:
• Percentage of health facilities with a skilled provider
• Percentage of health facilities with an adequate supply of
drugs
Some health indicators can be reliably measured with a
household survey or health-facility survey but not with
data reported routinely by health facilities:
• Proportion of population using an improved drinking water
source
• Proportion of health facilities with treatment guidelines
• Infant mortality rate
Some indicators are not clearly defined and usually cannot
be reliably measured with data reported routinely by
health facilities:
9
• % of health facilities with health equipment
Practicing the First Step
Exercise: Part 1
• Review the list of core health indicators (on
the next slide (Handout 5.2.1)
• Identify which indicators can/cannot be
reliably measured with routine data
o Can the numerator be defined with routine
data?
o Do you need additional data sources to
measure the indicator?
10
List of Health Indicators
1. Percentage of pregnant women 9. Percentage of HIV-positive infants
attending antenatal clinics who born to HIV-positive women
are screened for syphilis
10. Maternal mortality ratio
2. Neonatal mortality rate
11. Number of health facilities
3. Number of neonatal tetanus providing comprehensive
cases emergency obstetric care
functions per 500,000 population
4. Cost of one month’s supply of
contraception as a percentage of 12. Exclusive breastfeeding rate
monthly wages
13. TB treatment success rate
5. Percentage of infants born
protected against neonatal 14. Percentage of health facilities with
tetanus in a specified period systems that support quality
service delivery
6. Measles vaccine coverage rate
15. Percentage of districts with
7. Percentage of registered new and current trend analysis for selected
relapse tuberculosis (TB) patients priority diseases at a given time
with documented HIV status (e.g., month)
8. Percentage of children ages 12–
59 months who were dewormed
in the past six months 11
S1: Practicing the First Step
• Exercise, Part 2
• Review the WHO’s Global Reference List of
Core Health Indicators (Handout 5.2.3) or the
list of standard indicators in Handout 5.2.2.
• Select five indicators from the list.
• For each indicator:
o Specify the numerator
o Specify the denominator
o What is the data source for the numerator?
o What is the data source for the
denominator?
12
o How can the indicator be interpreted?
S2: Second Step
Review Data Quality
13
Step 2: Review Data Quality (see also Module 4)
• Completeness and timeliness
o Completeness of reports
o Completeness of data
o Timeliness of reports
• Internal consistency
o Accuracy
o Outliers
o Trends
o Consistency among indicators
• External consistency
o Data triangulation
o Comparison with data surveys
o Consistency of population trends
• External comparisons (population
14
denominators)
S2: Demonstration, Excel
How to use Excel to:
• Create a chart
Participants are invited to practice
simultaneously.
Here are two websites with 5-minute videos explaining
how to create a chart using Excel:
https://www.youtube.com/watch?v=BcsDnRClzCY
http
://excelcentral.com/excel2007/essential/lessons/05010-cr
eate-a-simple-chart-with-two-clicks.html
15
S2: Demonstration DHIS 2
How to use DHIS 2 to:
• Verify the definition of an indicator
o Numerator and denominator
• Create a pivot table
• Create a chart
Participants are invited to practice
simultaneously.
16
S3: Third Step
Select Appropriate Denominators
17
S3: How Do We Get Coverage Denominators?
• First, estimate the size of the target population.
• Common target populations for health-facility-based indicators:
o Total population, children< 5 years, infants, pregnancies,
women of reproductive age, live births at health facilities
• Size of target populations is often estimated (such as
projections/modeled estimates from national population
census).
• Limitations of estimates:
o Reliability declines with years since last census
o Internal migration may make estimates of populations of
regions and districts unreliable
• Some programs may use their own denominators.
18
How Do We Get Denominators?
• Document how the denominator value was obtained:
o Methods and assumptions used to calculate the
denominator
o Annual rate of growth if denominators are based on
projections of census figures
o Present these along with rest of analysis
• If service coverage >= 95% and data are of high quality,
use ANC1 or DTP1 to estimate the number of surviving
infants.
o Use of service statistics to estimate size of target
population can modify conclusions reached about which
districts are strong performers and which are weak
performers.
19
Estimating Denominators
Estimating the number of surviving infants:
Total population: 5,500,000
Crude birth rate (CBR): 30/1,000
Infant mortality rate (IMR): 80/1,000
Number of surviving infants
Total population x crude birth rate x (1 - IMR)
= 5,500,000 x 30/1000 x (1 - 0.080)
= 5,500,000 x 0.030 x
0.920
= 151,800
20
Data Quality Check for Denominators
• Pregnancies = births + pregnancy loss (2% to 10%);
• Surviving infants = births – infant mortality
Number of pregnancies, deliveries, live
births, infants
14000
Tanzania district example, 2014
12000 12415
11512 11286 11400
10000 11058
8000
6000
4000
2000
0
Pregnancies, Pregnancies, Deliveries Live births Infants
early late 21
S4: Fourth Step
Reconcile Findings with Estimates from Other Data
Sources
22
S4: Reconcile Findings with Estimates from
Other Data Sources
• Compare data from parallel systems that
routinely report the same health events.
• Compare estimates from routine health-facility
data with estimates from household surveys at
the national and regional levels.
• Compare available data with statistics that
have been officially reported to WHO.
23
S4: Estimates from Two Sources of Routine
Health Facility Data
Administrative estimates of 2014 DTP3 coverage, by district,
DPI/JRF versus HMIS/DHIS data
140%
120%
DTP3 -- HMIS (DHIS)
100%
80%
60%
40%
.
40% 60% 80% 100% 120% 140%
DTP3 - EPI (JRF)
Note: Red marks are for districts with a negative dropout rate, according to EPI/JRF data.
S4: Compare Estimates from Household
Surveys
DTP3 immunization coverage, Tanzania, 2009–2012
Sources: Routine EPI facility data, 2010 DHS, and 2011 Immunization
Coverage Survey 25
Reconciling with Survey Findings Is Not
Always Easy
2014 WHO-UNICEF report on trends in DTP3 coverage in
Ethiopia
Administrative/official estimates (red stars and circles) versus
surveys (vertical red lines) versus WHO-UNICEF estimate (blue
line)
26
Reconciling with Survey Findings Is Not
Always Easy
2014 WHO-UNICEF report on trends in DTP3 coverage in
India
Administrative/official estimates (red stars and circles) versus
surveys (vertical red lines) versus WHO-UNICEF estimate (blue
line)
27
Small Group Exercise (Steps 1-4 )
• Distribute Handouts 5.2.4a and b.
• Form small groups of 4–5 participants.
• Using Excel and the spreadsheet provided (Penta, 2014 data
from Tanzania, Handout 5.2.4B), calculate indicator values by
region for:
o Penta1 coverage rate
o Penta3 coverage rate
o Penta1-Penta3 dropout rate
• For each region, specify whether access is good or poor.
• For each region, specify whether utilization is good or poor.
• Categorize the immunization problem in each region (if any).
• Brainstorm the differences in coverage between regions.
• Discuss what action managers can take if coverage and dropout
rates indicate problems.
28
P2: Small Group Exercise, Option B
(DHIS 2)
• Distribute Handout 5.2.6.
• Form small groups of 4–5 participants.
• Choose either ANC4 or DTP3 as your indicator.
• Using DHIS 2, verify the definition of the indicator.
• Create a pivot table with indicator values by district.
• Graph trends in the completeness of reporting by year
for the past four years by:
o Facility (hospital versus health center) and
o Managing authority (public versus private)
• Discuss how the trend in completeness would affect
the apparent trend in the indicator.
29
ROUTINE HEALTH INFORMATION SYSTEMS
A Curriculum on Basic Concepts and Practice
This presentation was produced with the support of the United States Agency for
International Development (USAID) under the terms of MEASURE Evaluation
cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is
implemented by the Carolina Population Center, University of North Carolina at
Chapel Hill in partnership with ICF International; John Snow, Inc.; Management
Sciences for Health; Palladium; and Tulane University. The views expressed in this
presentation do not necessarily reflect the views of USAID or the United States
government.
30
