Scientific Review of the Proposed
Risk Assessment Bulletin from the
Office of Management and Budget
Committee to Review the OMB Risk Assessment Bulletin
Board on Environmental Studies and Toxicology
Division on Earth and Life Studies
February 2007
National Research Council
of the National Academies
The National Academies Press
Washington, DC
www.npu.edu/catalog/11811.html
Excerpt on Risk Comparisons (pp. 37-38)
RISK COMPARISIONS
There are two conceivable legitimate purposes for risk comparisons. Readers who
consult the risk communication literature will find that serving either purpose requires
both formal analysis to ensure that defensible comparisons are being made and dedicated
empirical research to ensure that the result is understood as intended. Readers of that
literature will also find that poorly done risk comparisons can confuse, mislead, and
antagonize recipients. Unless done in a scientifically sound way, risk comparisons are
unlikely to be useful and relevant and hence should be avoided.
One conceivable legitimate purpose is giving recipients an intuitive feeling for
just how large a risk is by comparing it with another, otherwise similar, risk that
recipients understand. For example, roughly one American in a million dies from
lightning in an average year (NOAA 1995). “As likely as being hit by lightning” would
be a relevant and useful comparison for someone who has an accurate intuitive feeling for
the probability of being hit by lightning, faces roughly that “average” risk, and considers
the comparison risk to be like death by lightning in all important respects. It is not hard to
imagine each of these conditions failing, rendering the comparisons irrelevant or harmful:
(a) Lightning deaths are so vivid and newsworthy that they might be
overestimated relative to other, equally probable events. But “being struck by lightning”
is an iconic very-low-probability risk, meaning that it might be underestimated. Where
either occurs, the comparison will mislead (Lichtenstein et al. 1978; NRC 1989).
(b) Individual Americans face different risks from lightning. For example, they
are, on the average, much higher for golfers than for nursing-home residents. A blanket
statement would mislead readers who did not think about this variability and what their
risk is relative to that of the average American (Slovic 2000; Tversky and Kahneman
1974).
(c) Death by lightning has distinctive properties. It is sometimes immediate,
sometimes preceded by painful suffering. It can leave victims and their survivors
unprepared. It offers some possibility of risk reduction, which people may understand to
some degree. It poses an acute threat at some very limited times but typically no threat at
all. Each of those properties may lead people to judge them differently— and undermine
the relevance of comparisons with risks having different properties (Fischhoff et al. 1978;
Lowrance 1976).
(d) It is often assumed that the risks being used for comparison are widely
considered acceptable at their present levels. The risks may be accepted in the trivial
sense that people are, in fact, living with them. But that does not make them acceptable in
the sense that people believe that they are as low as they should or could be. It would be
wrong to make comparisons with risks that responsible organizations are working
diligently to reduce. For example, the National Lightning Safety Institute (NLSI) and the
United States Golf Association do not consider contemporary risks of injury and death
from lightning strikes to be acceptable: “A strong case can be made for reducing
lightning’s human and economic costs through the adoption of proactive defensive
guidelines” (Kithil 1995).
The second conceivable use of risk comparisons is to facilitate making consistent
decisions regarding different risks. Other things being equal, one would want similar
risks from different sources to be treated the same. However, many things might need to
be held equal, including the various properties of risks (discussed above) that might make
people want to treat them differently despite similarity in one dimension (for example,
annual fatality rate among Americans) (HM Treasury 2005; Wittenberg et al. 2003).
The same risk may be acceptable in one setting but not another if the associated
benefits are different (for example, being struck by lightning while golfing or working on
a road crew). Even when making voluntary decisions, people do not accept risks in
isolation but in the context of the associated benefits. As a result, acceptable risk is a
misnomer except as shorthand for a voluntarily assumed risk accompanied by acceptable
benefits (Fischhoff et al. 1981).
The bulletin does not convey how difficult it is to produce useful and relevant risk
comparisons. Unless such comparisons are developed in a scientifically sound and
empirically evaluated way that addresses the values and circumstances of all recipients,
risk comparisons should not be made.
REFERENCES
Fischhoff, B., P. Slovic, S. Lichtenstein, S. Read, and B. Combs. 1978. How safe is safe
enough? A psychometric study of attitudes towards technological risks and
benefits. Policy Sci. 9(2): 127-152.
Fischhoff, B., S. Lichtenstein, P. Slovic, S.L. Derby, and R.L. Keeney. 1981. Acceptable
Risk. New York: Cambridge University Press.
HM Treasury (Her Majesty’s Treasury). 2005. Managing Risks to the Public: Appraisal
Guidance. London: HM Treasury. June 2005: http://www.hmtreasury.
gov.uk/media/8AB/54/Managing_risks_to_the_public.pdf
Kithil, R. 1995. Lightning's Social and Economic Costs. Presentation at International
Aerospace and Ground Conference on Lightning and Static Electricity, September
26-28, 1995, Williamsburg, VA [online]. Available:
http://www.lightningsafety.com/nlsi_lls/sec.html
Lichtenstein, S., P. Slovic, B. Fischhoff, M. Layman, and B. Combs. 1978. Judged
frequency of lethal events. J. Exp. Psychol. Learn. 4:551-578.
Lowrance, W.W. 1976. Of Acceptable Risk: Science and the Determination of Safety.
Los Altos, CA: W. Kaufmann.
NOAA (National Oceanic and Atmospheric Administration). 1995. Natural Hazard
Fatalities for the United States, 1994. National Oceanic and Atmospheric
Administration, Washington, DC.
NRC (National Research Council). 1989. Improving Risk Communication. Washington
DC: National Academy Press.
Slovic, P. 2000. The Perception of Risk. London: Earthscan.
Tversky, A., and D. Kahneman. 1974. Judgment under uncertainty: Heuristics and biases.
Science 185(4157):1124-1131.
Wittenberg, E., S.J. Goldie, B. Fischhoff, and J.D. Graham. 2003. Rationing decisions
and individual responsibility in illness: Are all lives equal? Med. Decis. Making
23(3):194-221.
