EPID: Focus on Surveillance
Novel Surveillance Systems: Good Value for the Money Spent?**
Arthur L. Reingold, M.D., M.P.H.
Professor, Division of Epidemiology
Associate Dean for Research, School of Public Health
University of California, Berkeley
“Syndromic surveillance” (surveillance for clinical syndromes without or before a definitive diagnosis can be or is made) as established and being utilized in various locations in the United States since 2001 to facilitate earlier detection and/or more targeted responses to outbreaks of disease is of unproven benefit. It is time to subject syndromic surveillance to rigorous evaluation to determine whether it provides benefits to public health or national security. if its benefits to public health and society cannot be documented, the federal resources currently devoted to it should be redirected to other disease surveillance approaches of proven worth.
As is well known to the public health community, the tragic events of September 2001, led to an enormous infusion of (largely federal) funds to support implementation of and research concerning novel methods of disease surveillance in the U.S., in addition to monitoring of environmental samples for detection of microbial agents posing a threat to human health. These novel methods of disease surveillance, often collectively referred to as “syndromic surveillance,” typically involve monitoring the frequency of one or more illnesses with a specified set of clinical features, such as diarrhea or signs and symptoms of respiratory tract infections in a population defined either geographically; by membership in a group like a health maintenance organization or the military, or perhaps simply by choice of health care provider. Not only have such systems been put into place in many locations and settings, but investigators at numerous academic and research institutions have established entire research programs dedicated to examining how well such systems do or might perform, especially with regard to “signal detection,” (i.e., the identification of outbreaks, whether naturally-occurring or produced intentionally). One might go so far as to say that a “syndromic surveillance” industry has been spawned, including academic journals and annual scientific meetings dedicated in whole or in large part to the study of “syndromic surveillance” in its various forms.
There has never been serious reason to doubt that, once certain statistical parameters have been stipulated, even a fairly rudimentary surveillance system could detect time-space clusters of illnesses of sufficient size. However, the practical utility of such systems in the “real world” of public health has been questioned (Reingold, 2003). While much of the impetus to develop and continue to fund such systems derived from a concern about bioterrorist events, such as the intentional release of B. anthracis spores in the U.S. in 2001, even the most ardent supporters of “syndromic surveillance” generally have conceded that such systems could not have detected that event. Many experts also acknowledge that such systems are unlikely to prove their worth through the earlier detection of some future bioterrorist event, if it ever occurs. As a result, much (perhaps all) of the value of such systems is generally purported to rest on the more rapid identification and/or characterization of naturally-occurring outbreaks of disease, so as to allow or facilitate either more rapid or better-targeted responses and presumably yield reductions in morbidity and/or mortality. However, in the “real world” of public health, all or virtually all outbreaks of any significance are detected through existing, disease-specific surveillance systems and/or reporting by astute clinicians or laboratorians, and it is very unlikely that syndromic surveillance will detect important outbreaks that would otherwise go undetected. Earlier detection of some outbreaks, while possible, will rarely lead to reduced morbidity or mortality, because useful interventions either do not exist or will still arrive too late. And, the substantial burden of the work required to follow-up on “signals” (i.e., possible increases in the incidence of a given syndrome in a given population during a given time period) and determine both the significance of the “signal” and its cause invariably falls on already overworked and understaffed public health agencies.
There is experience using “syndromic surveillance” to detect and facilitate the prompt response to epidemics of influenza, meningococcal meningitis, and other illnesses that substantially predates the events of 2001. However, the infusion of substantial resources into the development and evaluation of such systems, as well as into theoretical research (e.g., computer simulations that “inject” an artificial “event” into a real or manufactured database) concerning the likely performance characteristics of such systems has led to an outpouring of papers and presentations over the past six to eight years, particularly in the U.S. and, to a lesser extent, in other developed countries.
These same nations have also begun to make substantial resources available to improve disease surveillance and outbreak detection capabilities in selected developing countries. The motivations for improving disease monitoring in developing countries include the recognition that many new, “emerging” infectious diseases are zoonotic in origin and originate in Africa, Southeast Asia, and South America. However, the challenges and costs of implementing and sustaining disease surveillance activities in developing countries are substantial, and the utility of “syndromic surveillance” and other novel surveillance modalities in such settings also remains unproven.
Scientific opportunities and challenges
Given the very substantial financial resources invested in the implementation of and research concerning “syndromic surveillance,” I believe it would be timely and appropriate to conduct a comprehensive review of what these systems have accomplished and whether these resources might profitably be redirected in the future toward strengthening other, more mundane forms of disease surveillance in the U.S. One example of an alternative use of such funds to strengthen disease surveillance in the U.S. comes from the network of Emerging Infections Programs (EIPs) funded by the U.S. Centers for Disease Control and Prevention (CDC) since 1995. (Editor’s note: The author has been the director of the California EIP site since its inception in 1995).
Originally comprised of two sites in the U.S., the EIP network currently includes 10 sites with approximately 44 million people under surveillance, although the number varies by disease for which surveillance is conducted. The EIP network conducts enhanced, population-based surveillance for a variety of foodborne infections (FoodNet), invasive infections caused by selected bacterial pathogens (ABCs), influenza, and other high-priority infectious diseases. Specially hired, dedicated staff conduct surveillance using a common protocol, including collection of bacterial and viral specimens as well as periodic laboratory audits to ensure the accuracy of laboratory-confirmed cases.
The surveillance system also provides the infrastructure underpinning a wide variety of analytic epidemiologic and laboratory studies of diverse infectious diseases. To date, the EIP has produced more than 500 publications concerning the various diseases under surveillance, many of them landmark studies. The EIP network, with the supplemental federal funds provided to it by CDC (at a cost of approximately $500,000 per million people under surveillance), is able to conduct disease surveillance of a quality, completeness, and consistency not generally attainable by state and local health departments that must rely solely on state and local funding, especially given the extent to which such funding rises and falls with economic conditions.
An alternative use of federal funds currently used to support “syndromic surveillance” implementation and research would be to finance a national system of disease surveillance based on the EIP model that has assured funding and that uses a common set of protocols to assure consistent, complete, and timely disease reporting. Given that disease surveillance in the U.S. is a state and local, rather than a federal mandate, such a system might be based on the federal highway system — with states having to adhere to federally established protocols, including those for data collection, storage and transmission, as well as identifier-free data-sharing with the CDC, to receive federal funds.
- Unless “syndromic surveillance” can be shown to produce good value for the money spent, an alternative use of federal funds to support enhanced disease surveillance activities in the U.S. and elsewhere should be given serious consideration.
- Federal agencies that provide financial support to “syndromic surveillance” activities in the U.S. should rigorously assess the concrete outputs of such activities with regard to outbreaks detected; morbidity and mortality averted, and other purported benefits.
- An alternative use of federal funds for disease surveillance would be to implement a national system modeled after the existing Emerging Infections Programs funded by CDC. Similar to the national highway system, this national surveillance system would be paid for by the federal government, but implemented by the state and territorial agencies with the legal responsibility to conduct disease surveillance, using methods, data collection instruments, and case definitions set forth by the CDC, after consultation with state and territorial agency partners.
- Internationally, similarly rigorous evaluations of the utility of various approaches to disease surveillance must be conducted, and the World Health Organization funded and empowered to oversee and coordinate such efforts.
Reingold A.L. (2003). If syndromic surveillance is the answer, what is the question? Biosecurity & Bioterrorism, 2003; 2:1-5.
** A policy position paper prepared for presentation at the conference on Emerging and Persistent Infectious Diseases (EPID): Focus on Surveillance convened by the Institute on Science for Global Policy (ISGP) Oct. 17-20, 2010, at Airlie Conference Center, Warrenton, Va.
The following summary is based on notes recorded by the ISGP staff during the not-for-attribution debate of the policy position paper prepared by Prof. Arthur Reingold (see above). Prof. Reingold initiated the debate with a 5-minute statement of his views and then actively engaged the conference participants, including other authors, throughout the remainder of the 90-minute period. This Debate Summary represents the ISGP’s best effort to accurately capture the comments offered and questions posed by all participants, as well as those responses made by Prof. Reingold. Given the not-for-attribution format of the debate, the views comprising this summary do not necessarily represent the views of Prof. Reingold, as evidenced by his policy position paper. Rather, it is, and should be read as, an overview of the areas of agreement and disagreement that emerged from all those participating in the critical debate.
- The definition, utility, and value of syndromic surveillance all remain under debate. The usefulness of syndromic surveillance in designing effective treatments for infectious disease outbreaks was recognized, especially if syndromic information is combined with other data such as laboratory diagnostics. The effectiveness of data from syndromic surveillance as a separate tool for early warning remains controversial.
- Extensive, but not conclusive, discussion focused on the priorities with which resources are allocated to syndromic surveillance relative to other approaches used for surveillance of infectious diseases. Significant disagreement was expressed over the current importance of syndromic data in helping to shape public policy on identifying and controlling infectious diseases, and accordingly little support was expressed for reducing current resources allocated to syndromic surveillance.
- The value of syndromic surveillance as a deterrent against potential terrorist activities related to infectious diseases was noted.
- Historically, emerging diseases have been recognized by individuals noticing health conditions that differ from those ordinarily encountered. The rapid expansion of information technology has made it feasible to routinely share large amounts of information concerning the general health of communities, which therefore facilitates recognition of unusual events.
From the outset of the discussion, attention was focused on whether syndromic surveillance has much practical utility for early warning with respect to infectious diseases. The question of whether there is an adequate return on the current financial and human investments allocated to syndromic surveillance was also raised. The relevance of these questions to more and less-wealthy countries was an important part of the discussion.
Although syndromic surveillance was initially emphasized as a biodefense measure after the 2001 anthrax attacks in the United States, there was little support for the conclusion that this type of surveillance would have detected those attacks. Nor was there support for the value of syndromic surveillance as a predictor for almost any of the instances of bioterrorism in recent history in the U.S. There was considerable discussion concerning the assertion that the U.S. federal government needs to re-evaluate the role of syndromic surveillance and redirect its syndromic surveillance funding elsewhere. Such a decision in the U.S. would likely have a direct influence on the role of and support for syndromic surveillance worldwide.
There was consensus that syndromic surveillance is not the optimal surveillance system. However, many challenged the assertion that syndromic surveillance has no public health value and that funds should be directed elsewhere. Instances where syndromic surveillance proved to be useful were cited. For example, it was noted that syndromic data were used to provide a clinical definition of smallpox during the smallpox eradication effort, a definition that replaced laboratory confirmation as a method for identifying cases. In addition, it was asserted that the initial detection of SARS involved data on the symptoms of patients (i.e., syndromic surveillance) that facilitated global recognition of the disease at an early stage. It was conceded that syndromic surveillance was useful in the general context of surveillance for some specific diseases with well-defined intervention programs. For example, syndromic surveillance for acute flaccid paralysis was recognized as an effective method to identify probable polio cases. However, there are other diseases, such as malaria, for which syndromic surveillance is not effective. Syndromic surveillance using fever has not produced accurate or reliable information about malaria.
There was also agreement that syndromic surveillance has potential benefits in less-wealthy countries, where resources to conduct infectious disease surveillance through laboratory confirmation are generally not available. In those settings, syndromic surveillance can establish baselines against which the presence of disease outbreaks can be measured and thereby serve as a stop-gap method until adequate laboratory and epidemiologic capacity can be established.
In light of the varying uses for syndromic surveillance, consensus was reached that the benefit of this surveillance method should be evaluated on a case-by-case basis. Specific situations and circumstances exist where syndromic surveillance would have public health benefits that are commensurate with the costs involved. However, when more thorough, laboratory-based surveillance methods are available, syndromic surveillance may not provide the best investment of funding. There was some agreement that many millions of dollars are currently spent annually on syndromic surveillance in the U.S., although no specific number was confirmed.
Scientific opportunities and challenges
After extensive discussion, there remained questions about what constitutes a working definition for syndromic surveillance. The absence of a widely accepted definition was viewed as a barrier to determining how effective its role can be in surveillance systems. Myriad examples of syndromic surveillance were provided, including looking at newspaper reports, Web search queries, and recording malaria cases via public cell phones. There was agreement that a widely accepted definition for syndromic surveillance would help avoid confusion and misunderstanding about its value in disease surveillance.
Scientific capabilities and information technology have progressed so rapidly that there is now a voluminous amount of data covering a wide range of information that can be considered syndromic surveillance. While technological advancements are likely to keep moving forward, the challenge becomes how to accurately analyze this increasingly diverse information in ways that make it valuable to the decision maker. It was noted that syndromic surveillance itself can be viewed as its own new technology and that with time and experience, it may mature into an inexpensive and useful surveillance technique that stands alone. This point was questioned by refocusing on whether having more and more data enhances understanding or merely complicates the challenges of analysis (i.e., can the most important data be found when the amount of data continues to increase rapidly?).
Even where syndromic surveillance is successful in identifying outbreaks, concern was expressed regarding whether this information alone would translate into effective preventive action. The example of influenza was used to illustrate that earlier notification of an influenza outbreak solely from syndromic data did not lead health agencies to begin activities that would have shortened the influenza epidemic. Counter to this argument were many comments describing the benefits of advanced notice of an outbreak to target vaccination efforts and increase public health messaging in affected areas. These latter activities could have a significant positive effect on controlling, or at least lessening, the impact of an influenza outbreak.
Although the value of syndromic surveillance for advanced warning of epidemics due to known pathogens remains unclear, there was consensus that the current syndromic methodologies would not be useful in identifying diseases associated with unknown pathogens. Historically, a new pathogen has been identified by an individual noticing something out of the ordinary from the normal health patterns. However, it was noted that syndromic surveillance could be a valuable tool for systematically capturing and recording unusual events that are often used to characterize the health of a community at a central location. This background information derived from syndromic surveillance can be valuable in recognizing disease outbreaks and properly analyzing their potential impact.
Even in the event that syndromic surveillance could be systematically defined and more widely accepted as a viable surveillance option for public health, the need for thorough evaluations and analyses of a surveillance system including syndromic methods would still exist. There was consensus that syndromic surveillance data alone likely would not be adequate to mobilize significant actions by policy makers who seek to combat a potential infectious disease outbreak. The identification of additional information concerning public health in general may provide a broader understanding of the presence and intensity of infectious diseases as part of an overarching surveillance system. In these cases, syndromic surveillance data can serve as a supplementary data source that can be useful to policy makers.
The policy issue concerning whether large amounts of funding should be allocated to syndromic surveillance, given its limited value to public health, remained controversial. It was generally agreed that syndromic surveillance data had limited public health applications and that policy makers should be aware of the limitations of this form of surveillance and be cautious about how this information is used in initiating specific actions. It was advocated that syndromic surveillance data be used in settings where a specific disease is targeted and where specific intervention activities are being considered, especially when little other data are available. Complete re-evaluation of syndromic surveillance programs in general and how public resources are being allocated is a worthy activity, but any reallocation of support away from syndromic surveillance needs to be carefully considered in terms of how effective alternative options can be in providing more reliable information.
Much discussion centered on the need to supplement syndromic surveillance with confirming data from laboratory analysis. This was seen as a challenge for countries with weak laboratory capacity. Although there was agreement on the need to increase laboratory capacity in developing countries to improve all surveillance efforts, including syndromic, there was little agreement on specific ways to proceed except with respect to providing more funding. There was overwhelming support for increased funding for laboratory capacity building in less-wealthy countries. It was agreed that optimal surveillance techniques for the investigation of infectious disease outbreaks require laboratory confirmation of diagnosis, and that such confirmation requires adequate laboratory capacity in terms of both equipment and personnel. The major obstacle for an increase in capacity was determined to be the lack of a dedicated funding stream to improve laboratories around the world.
The role of the World Health Organization (WHO) in evaluating surveillance was vigorously debated. The recommendation that the WHO be funded and empowered to engage in systematic and complete evaluations of international disease surveillance systems, including syndromic surveillance systems, was only partially supported. In this proposal, the WHO would have both advisory and coordinating roles in such evaluations. Serious concerns were raised over whether the current WHO structure is appropriate to perform these evaluations.
The interests of policy communities in the data obtained through syndromic surveillance can be different from the interests of the public health community. Although biodefense syndromic surveillance programs do not normally provide direct public health benefits, they can serve as a deterrent against those who would consider an attack on the public. Thus, syndromic surveillance can have multiple benefits that span the interests of both the public health and national security communities.
Copyright: Institute on Science for Global Policy