Introduction to EBM

Rodney Smith, MD
St. Joseph Mercy Hospital
University of Michigan
Objectives
 What is EBM and Why do it?
 Describe the EBM Process and its
components
 Patient Problem
 Clinical question
 Search for Evidence
 Critical Appraisal of the Evidence
 Apply the Results

 The EBM Lecture

Introduction
 What is EBM?
 "...the conscientious, explicit, and judicious use of
current best evidence in making decisions about the
care of individual patients."
 Integrated with clinical expertise
 expertise in performing the history and physical
examination
 knowledge of the patient, the family, and the
community which creates a context for therapeutic
decision-making
 a relationship with the patient informed by his or her
beliefs and values
 practical knowledge of the availability of resources in
the community
Introduction
Introduction
 "Without clinical expertise, practice risks
becoming tyrannized by external evidence,
for even excellent external evidence may be
inapplicable to or inappropriate for an
individual patient. Without current best
external evidence, practice risks becoming
rapidly out of date, to the detriment of
patients."
Introduction
 Why do EBM
 It is required by RRC/Residency
 Provide high quality care to patients
 Utilize high quality evidence in patient care
 Overcome limitations of current practice
Introduction
 Examples
 Prevention of SIDS…”Back to Sleep”
 Patching corneal abrasion
 Arryhthmia suppression
 Ecainide/flecainide
 Arrhythmia suppression
 RCT increased mortality
 Disease specific vs. patient specific outcomes
Introduction
 The failure of common sense
 I shake a tree, an apple falls out, now I can eat
 I give antibiotics for sinusitis, the patient gets
better, I’m doing a good job as a doctor
 I just saw a patient with tumor lysis syndrome,
I will start looking for it more and maybe I’ll
find it more
 Introduction
 Variation in practice
 Rate of prostatectomy
for prostate CA
Introduction
Introduction
 Source of Medical Information
 Colleagues
 Conferences
 Drug Reps
 Textbooks
 Journals
 Internet/Patients
Introduction
The Clinical Question
 Physician recall
 1 question per 4 patients
 Direct observation Covell DG, et al. Ann Intern Med
1985;103:596-9.
 2 questions per 3 patients
 15 questions per shift
 Information sources

Textbooks

PDR

Human sources
 2/3 of questions go unanswered
The Clinical Question
 Unrecognized
 Recognized
 Pursued
 Satisfied
 Implemented
The Clinical Question
The Question
 Background
 Anatomy and Physiology
 Pathophysiology
 Pharmacology and Toxicology
 Differential diagnosis
 Diagnostic testing
 Treatment
 Textbooks, reviews, lectures, experts
The Clinical Question
 Foreground
 Detailed information
 Patient focus
 Evidence-based process
The Clinical Question
 Patient or problem
 Intervention
 Comparison
 Outcome
The Question
 Patient or Problem
 Starts from the patient encounter
 Unrecognized vs. Recognized

 Intervention
 Be specific
 Compared to what
 Outcome
 Disease-center vs. Patient-centered
Example
 68 yo WF, Hx CHF, S/P CABG, HTN, COPD
 Meds: Digoxin, Lasix, Isosorbide, Albuterol
 Mild increase in DOE past 4 days
 Acutely SOB 1hr PTA
 188/104 122 30 98.8 90% Non-rebreather
 Crackles bases, Wheezes scattered
 JVD, S3 gallop
 Pretibial edema 2+
Example
 BNP now available at St. Joe’s
 Should I order a BNP on this patient?
The Clinical Question
 Patient or problem
 Intervention
 Comparison
 Outcome
The Question
 ED patients with acute dyspnea
 BNP
 Standard evaluation
 Diagnosis of CHF
Finding the Evidence
 Textbooks – NOT!
 Always out of date
 Recommendations often not referenced
Finding the Evidence
 Textbooks
 Clinical Evidence
 Published twice yearly
 Full version 1900 pages
 Concise version 400 pages
 CD ROM
 Online Access
 Cost $110
 www.clinicalevidence.com
Finding the Evidence
 Textbooks
 Scientific American Medicine
 Online subscription $199
 www.samed.com
 Harrison’s Textbook of Internal Medicine
 Online subscription $99
 www.harrisonsonline.com
Finding the Evidence
 EBM Textbook
 Evidence-Based On-Call
 Database of critically appraised topics (CATs)
 Bullet points with links to in-depth CAT
 www.eboncall.co.uk/content.jsp.htm
Finding the Evidence
 EBM Journals
 ACP Journal Club
 6 Issues per year
 Online access
 $78/yr Individual
 $55/yr Residents
Finding the Evidence
 EBM Journals
 Bandolier
 Summaries of secondary reviews
 Free online access
 www.jr2.ox.ac.uk/bandolier
Finding the Evidence
 EBM Journals
 Evidence Based Medicine
 Now combined with ACP Journal Club
 Articles from 1999 and prior available free online
 www.acponline.org/journals/ebm/ebmmenu.htm
Finding the Evidence
 EBM Databases
 Evidence Based Medicine Reviews (EBMR)
 Commercially available thru OVID
 Searches a variety of databases
 Cochrane Database of Systematic Reviews
 ACP Journal Club
 Medline
 Links databases together
Finding the Evidence
 EBM Databases
 Cochrane Library
 The Cochrane Database of Systematic Reviews
 The Cochrane Controlled Trials Register
Finding the Evidence
 EBM Databases
 SUMSearch
 Merck Manual.

 MEDLINE for review articles and editorials from high

quality, general journals that have full texts available.

 National Guideline Clearinghouse from the Agency for

Health Care Policy and Research (AHCPR)

 Database of Abstract of Reviews of Effectiveness

(DARE)
 MEDLINE for original research

 sumsearch.uthscsa.edu
Finding the Evidence
 EBM Databases
 MEDLINE
 Free online access thru PubMed
 www.ncbi.nlm.nih.gov/entrez/
 Search by MESH terms or free text
 EBM Filters
Finding the Evidence
Finding the Evidence
Finding the Evidence
Finding the Evidence
Finding the Evidence
Finding the Evidence
Finding the Evidence
 Filters for Medline Search
 Treatment
 Randomized controlled trial
 Blind or Double Blind
 Diagnosis
 Sensitivity and Specificity
Critical Appraisal
 Is the evidence valid?
 Is evidence important?
 Does the evidence apply to our patient?
Critical Appraisal
 Users’ Guides to the Medical Literature
 Book by Guyatt and Rennie
 Available thru AMA/ Amazon.com $35/$60
 CD-ROM version
 www.usersguides.org
 www.cche.net/usersguides/main.asp/
Critical Appraisal
 Diagnosis
 Is the evidence valid?
 Was there an independent, blinded comparison
with a gold standard?
 Was the test evaluated in an appropriate
spectrum of patients?
 Was the reference standard applied regardless of
the test result?
 Was the test validated in a second, independent
group of patients?
Critical Appraisal
 Diagnosis
 Is this valid test important?
 Distinguish between patients with and those
without the disease
 Two by two tables
 Sensitivity and Specificity
 SnNOut
 SpPIn
 ROC curves
 Likelihood Ratio
2 X 2 Table
 Sensitivity = 90%
 Specificity = 90%
Disease
 Pos Predictive Value =
90%
 Neg Predictive Value =
Present Absent
90%

Test Pos 90 10

Test Neg 10 90
2 X 2 Table
 Sensitivity = 90%
 Specificity = 90%
Disease
 Pos Predictive Value =
8.3% (90/1090)
 Neg Predictive Value =
Present Absent
99.9% (9000/9010)
 Prevalence = 1%
Test Pos 90 1000

Test Neg 10 9000

2 by 2 Table
 90% with disease have
a positive test
 15% without disease Disease

have a positive test

 Someone with a Pos
Present Absent
test is 6X more likely to
have the disease as not
(90%/15%) Test Pos 90 15 105
 Likelihood Ratio +
 Sens /(1- Spec) Test Neg 10 85 95

100 100 200

2 by 2 Table
 10% with disease have
a positive test
Disease
 85% without disease
have a positive test
 Someone with a Neg Present Absent
test is less likely to have
the disease by 1/8.5 X Test Pos 90 15 105
(10%/85%)
 Likelihood Ratio - Test Neg 10 85 95
 (1-sens)/spec
100 100 200
2 X 2 Table
 Pre-test Odds X LR = Post-test Odds
 Pre-test Odds =
Pretest Prob/(1-Pretest Prob)
 Post-test Probability =
Posttest Odds/(1 + Posttest Odds)
Probability to Odds

Probability Odds Decimal

1/2 1:1 1.0

1/3 1:2 0.5

3/4 3:1 3.0

0.45 0.45:0.55 0.8

Odds to Probability

Odds Probability

0.5 :1 0.33

0.25 :1 0.20

2.0 :1 0.66

10.0 :1 0.91
Critical Appraisal
 Diagnosis
 Can I apply this test to a specific patient
 Is it available
 Is it affordable
 Is it accurate
 Is it precise
Critical Appraisal
 Diagnosis
 Can I apply this test to a specific patient
 Can I generate a sensible pre-test probability
 Personal experience
 Practice database
 Assume prevalence in the study
Critical Appraisal
 Diagnosis
 Can I apply this test to a specific patient
 Will the post-test probability affect management

 Movement above treatment threshold

 Patient willing to undergo testing
Critical Appraisal
 Treatment
 Are the results of this study Valid?
 Were treatment assignments randomized and
concealed?
 Was follow-up sufficiently long?
 Were patients evaluated in the group to which
they were assigned?
 Was it double-blind?
 Were treatments equal apart from study
treatment?
 Were the groups similar at the start of treatment?
Critical Appraisal
 Therapy
 Are the valid results of this trial Important?
 What is the magnitude of the treatment effect?
 How precise is this estimate of the treatment
effect?
 95% CI
 Directly related to number of patients in a study
Therapy
 Are the valid results of this trial Important?
 What is the magnitude of the treatment effect?
 CER = control event rate
 EER = experimental event rate
 RRR = relative risk reduction

= |CER – EER|/CER
 ARR = absolute risk reduction

= |CER – EER|
 NNT = Number needed to treat

= 1 / ARR
Critical Appraisal

CER EER RRR ARR NNT

50% 25% 50% 25% 4

5% 2.50% 50% 2.50% 40

0.05% 0.025% 50% 0.03% 4000

Critical Appraisal
 Therapy
 Are the valid, important results applicable to
our patient?
 How similar is our patient to the patients studied?
 Is the treatment feasible?
 What are our patient’s potential benfits and
harms from the therapy?
 What are our patient’s values and expectations
for both the outcome we are trying to prevent and
the treatment we are offering?
 Critical Appraisal
 Therapy
CER EER RRR ARR NNT

Stroke or Death
5yr 50% - 69% 43% 33% 23% 10% 10

Stroke or Death
5yr <50% 37% 36.20% 2% 1% 126

Stroke or Death
30 Days 2.40% 6.700% 179% 4% 24
Summary of Evidence
 Critically Appraised Topic
 Bottom line, bullet points
 PICO question
 Search Terms
 Summary of paper and results table
 Comments
 Expiration date
 References
Using EBM in Emergency Medicine
 EBM Lectures
 Case conferences
 Core lectures
 Sign-out rounds
 The “EBM Prescription”
 Patient information
 The Clinical Question (PICO)

 Secondary sources
Introduction to EBM

Intermission
The Question
 ED patients with acute dyspnea
 BNP
 Standard evaluation
 Diagnosis of CHF
BNP Search
BNP Search
BNP Search
BNP Search
BNP Search
Critical Appraisal
Critical Appraisal
 Was there an independent, blinded
comparison with a gold standard?
 Clinical diagnosis of CHF
 Two independent cardiologists
 Clinical evidence available in medical record and
specific study data
 Blinded to results of BNP(ED MD blinded; ?
others blinded?)
 ? What if they disagreed ?
Critical Appraisal
 Was the test evaluated in an appropriate
spectrum of patients?
 1586 patients from 7 sites (5 US)
 Inclusion
 Shortness of breath as most prominent symptom
 Exclusion
 Under 18 years
 “dyspnea was clearly not secondary to congestive
heart failure (for example, those with trauma or
cardiac tamponade)”
 AMI or renal failure
 USA unless they met the inclusion criterion
Critical Appraisal
 Was the reference standard applied
regardless of the test result?
 Yes
 Was the test validated in a second,
independent group of patients?
 No
 BNP Results

BNP Sensitivity Specificity LR + LR-

50 97 62 2.55 0.05

80 93 74 3.58 0.09

100 90 76 3.75 0.13

125 87 79 4.14 0.16

150 85 83 5.00 0.18

BNP ROC Curve

LR + 10
BNP CAT Bottom Line
 47% of 1586 patients presenting to 7
emergency departments with dyspnea had
congestive heart failure
 A BNP less than 80 rules out CHF
(LR - = 0.09)
 For BNP above 150, LR + = 5.0
ACE Inhibitors in Acute
Pulmonary Edema

Introduction to
Evidence-Based Medicine
Rodney Smith, MD
Case Presentation
 68 yo WF, Hx CHF, S/P CABG, HTN
 Meds: Digoxin, Lasix, Isosorbide
 Mild increase in DOE past 4 days
 Acutely SOB 1hr PTA
 188/104 122 30 98.8 90% Non-rebreather
 Crackles to ½ bilat
 JVD, S3 gallop
 Pretibial edema 2+
Case Presentation
 IV, O2, Monitor
 Chest Xray, EKG, CCU labs, dig level
 EKG sinus tach, old iwmi, ns st-t changes
 CXR acute pulmonary edema
 Treatment
 Nitropaste
 Lasix
 MS
Question—General Statement
 What about using ACE inhibitors for acute
pulmonary edema?
Background
 Describe pathophysiology of CHF and acute
pulmonary edema
 Discuss causes of decompensation of CHF
 Discuss Diff Dx of acute pulmonary edema
 Relate pathophysiology of CHF to treatment,
especially role of ACE-I in CHF
 Describe treatment goals
 Describe standard treatment of CHF
EBM Question
 Patients: Acute Pulmonary Edema
 Intervention: ACE Inhibitor
 Comparison: Placebo
 Outcome:
 Mortality
 Intubation
 Hemodynamic parameters
 ICU/CCU admission
Critical Appraisal
 Placebo-controlled, randomized, double-blind study of
intravenous enalaprilat efficacy and safety in acute cardiogenic
pulmonary edema.
Circulation. 1996 Sep 15;94(6):1316-24
 ICU patients with Swan Ganz catheters
 Not intubated
 Improved with 6hr standard treatment
 Lasix
 Nitrates
 +/- Dopamine
Annane, et al.
Circulation. 1996;94(6):1316-24
 Six-hour washout period
 Randomized to IV enalapril or placebo
 Hemodynamic parameters measured
 Primary endpoints
 Pulmonary capillary wedge pressure
 Renal blood flow
Annane, et al.
Circulation. 1996;94(6):1316-24
 Statistically Significant Effects of ACE-I
 Decreased diastolic and mean arterial BP
 Decreased PCWP
 Improved Renal and Brachial blood flow
Annane, et al.
Circulation. 1996;94(6):1316-24
 Are the results Valid
 Assignment randomized
 Follow-up for 8 hours
 Intention to treat analysis
 Double blind
 Patients treated equally
 Groups were similar at start of treatment
Annane, et al.
Circulation. 1996;94(6):1316-24
 Are the valid results important?
 What is magnitude of treatment effect?
 PCWP

T0 T2 T4 T8
Enalaprilat 27 22 17 20

Placebo 21 21 19 22
Annane, et al.
Circulation. 1996;94(6):1316-24
 Are the valid results important?
 What is magnitude of treatment effect?
 Renal Blood Flow

T0 T4

Enalaprilat 617 690

Placebo 570 577

Annane, et al.
Circulation. 1996;94(6):1316-24
 Are the valid, important results applicable to
our patient?
 Patients in ICU, Swan Ganz, 6 hour pre-
treatment (after ED treatment)
 Treatment is feasible
 ?Potential benefits. No particular harms
 Patient values likely in favor of treatment if it is
of benefit
Critical Appraisal
 Rapid improvement of acute pulmonary edema with sublingual
captopril.
Acad Emerg Med. 1996 Mar;3(3):205-12
 Emergency department patients
 Randomized to sublingual captopril or
placebo (standard treatment in both)
 Outcomes
 Clinical acute pulmonary edema distress score
 Intubation
Hamilton RJ, et al.
Acad Emerg Med. 1996;3:205-12
 Improvement in APEX score
 Improvement in number intubated
 9% vs. 20% (NS, p = 0.1)
Hamilton RJ, et al.
Acad Emerg Med. 1996;3:205-12
 Are the results valid
 Randomized assignment
 Non-consecutive (57 of 107 enrolled)

 No data on 50 patients not enrolled


Too busy or research assistant not available

9 exclusions: 3 intubated, 5 incomplete records, 1
COPD
 Follow for 120 minutes
 Intention to treat analysis (see exclusions)
 Blinded to treatment (double blind)
 Equal treatment (“standard” therapy)
 Groups were similar at start of treatment
Hamilton RJ, et al.
Acad Emerg Med. 1996;3:205-12
 What is magnitude of
treatment effect?
 25 to 40 min
 APEX reduction of
57% in treated
 APEX reduction of
75% in placebo
 Absolute improvement
18%
Hamilton RJ, et al.
Acad Emerg Med. 1996;3:205-12
 What is magnitude of treatment effect?
 Intubation
 Absolute Risk Reduction: 20% - 9% = 11%
 95% CI -8% to 31%; p = .10
 Relative Risk Reduction: 11%/20% = 55%
 Number Needed to treat: 1/11% = 9
 Treat Nine patients to avoid One intubation
 With Confidence intervals
 NNH = 1/8% = 12
 NNT = 1/31% = 3
Hamilton RJ, et al.
Acad Emerg Med. 1996;3:205-12
 Are these valid, important results applicable
to our patient?
 Similar to our patient
 Treatment is feasible
 ?Potential benefit
 Patient Values likely in favor of treatment
Conclusion
 Patients with acute pulmonary edema treated
with sublingual captopril had 18% more
improvement at 30 minutes compared with
placebo
 Trend toward fewer intubations but study too
small to tell