Signal Detection, Uncertainty, and Value of Information in Testing and Evaluating Detection Technologies

Principal Investigator: 
Performance Period: 
February 2017 to June 2018
Project Status: 
In Progress
Commercialization Status: 
N/A
Abstract: 
Many DHS components have the task of detecting illegal materials and people. Examples are: 
  1.  DNDO is charged with detecting radiological and nuclear materials out of regulatory control on route from their source or when entering the United States
  2. CBP is charged with detecting drugs and illegal individuals entering the United States
  3. The US Coast Guard attempts to detect and interdict smugglers and drug traffickers
  4. The TSA aims at detecting guns and explosives carried by passengers onto airplanes
Signal detection theory was developed to evaluate detection technologies and to optimize their performance. It is the main model/tool used in this project. The project is a follow up on a relatively simple signal detection analysis to evaluate algorithms used by DNDO and CBP to detect radiological and nuclear materials at US Ports of Entry. This project lent support to new algorithms that drastically reduced the frequency of false alarms, while not leading to a reduction in detection rates. The follow up research concerned two questions:  What to do, when the false alarm rate and the detection rate are highly uncertain. The standard answer is:  Conduct more testing and evaluation to narrow down these rates. While this is often possible with false alarm rates (which are frequent and observable), the detection rates, especially of rare events remains uncertain, even with multiple tests. An important addition to signal detection theory is therefore a value of information analysis, which can provide an answer to the question: How much resources should be spent to reduce the uncertainties about the false alarm and the detection rate?