Communicating Probability in Intelligence Analysis and Homeland Security

Principal Investigator: 
Other Researchers: 
Jennifer Lerner, Jeffrey Friedman
Performance Period: 
July 2015 to June 2016
Project Status: 
In Progress
Commercialization Status: 
N/A
Project Keywords: 
risk communication
intelligence analysis
estimative probability
experiments
Abstract: 
Uncertainty surrounds virtually every major national security decision. This is especially true in homeland security, which explicitly focuses on assessing and mitigating risk. But unlike predicting the weather or calculating actuarial rates in insurance, national security analysts and decision makers rarely have objective ways to assess the uncertainties they confront. Over the last half century, national security officials have debated how best to communicate their assessments of uncertainty, particularly when it comes to communicating probability.[1] Numerical percentages are the clearest mode of expression, but many officials believe that they suggest unrealistic precision, which can distort analysis and decision making. The U.S. Intelligence Community (IC) thus  encourages officials to use seven “words of estimative probability,” as shown in Figure 1.[1] This guidance essentially represents an empirical claim about what works best, all else being equal, for communicating probability in national security analysis. Yet we are aware of no empirical research rigorously evaluating this subject in a national security context.[2] We propose to address this issue with survey experiments, using practitioners as respondents in order to maximize our ability to speak to national security analysis and decision making.   [1] Figure 1 appears in the front of National Intelligence Estimates. Though it is the closest thing to “doctrine” on the matter of expressing probability in national security, this guidance is not consistently implemented across agencies or outside of NIEs. [2] Previous research finds that context matters for determining the optimal method of expressing uncertainty, though unfortunately it offers few direct lessons for informing the practice of intelligence and homeland security. Our ideas on this subject have been shaped by the work of political scientists, economists, psychologists, and decision theorists, including William Boettcher, Wibecke Brun, David Budescu, Brent Cohen, Ido Erev, Baruch Fischhoff, Craig Fox, Claudia Gonzalez-Vallejo, Daniel Kahneman, Howard Kunreuther, Rose McDermott, Michael Olson, Ellen Peters, Paul Slovic, Karl Teigen, Amos Tversky, Thomas Wallsten and Paul Windschitl.