Purpose To develop and validate the positive predictive value (PPV) of an algorithm to identify anaphylaxis using health plan administrative and claims data. Previously published positive predictive values (PPVs) for anaphylaxis using ICD-9-CM codes range from 52-57%. Methods We conducted a retrospective study using administrative and claims data from eight health plans. Using diagnosis and procedure codes, we developed an algorithm to identify potential cases of anaphylaxis from the Mini-Sentinel Distributed Database between January 2009 and December 2010. A random sample of medical charts (N=150) was identified for chart abstraction. Two physician adjudicators reviewed each potential case. Using physician adjudicator judgments on whether the case met diagnostic criteria for anaphylaxis, we calculated a PPV for the algorithm. Results Of the 122 patients for whom complete charts were received, 77 were judged by physician adjudicators to have anaphylaxis. The PPV for the algorithm was 63.1% (95% CI: 53.9%-71.7%), using the clinical criteria by Sampson as the gold standard. The PPV was highest for inpatient encounters with ICD-9-CM codes of 995.0 or 999.4. By combining only the top performing ICD-9-CM codes, we identified an algorithm with a PPV of 75.0%, but only 66% of cases of anaphylaxis were identified using this modified algorithm. Conclusions The PPV for the ICD-9-CM-based algorithm for anaphylaxis was slightly higher than PPV estimates reported in prior studies, but remained low. We were able to identify an algorithm which optimized the PPV but demonstrated lower sensitivity for anaphylactic events.

Purpose To validate an algorithm based upon International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) codes for acute myocardial infarction (AMI) documented within the Mini-Sentinel Distributed Database (MSDD). Methods Using an ICD-9-CM-based algorithm (hospitalized patients with 410.x0 or 410.x1 in primary position), we identified a random sample of potential cases of AMI in 2009 from 4 Data Partners participating in the Mini-Sentinel Program. Cardiologist reviewers used information abstracted from hospital records to assess the likelihood of an AMI diagnosis based on criteria from the joint European Society of Cardiology and American College of Cardiology Global Task Force. Positive predictive values (PPVs) of the ICD-9-based algorithm were calculated. Results Of the 153 potential cases of AMI identified, hospital records for 143 (93%) were retrieved and abstracted. Overall, the PPV was 86.0% (95% confidence interval; 79.2%, 91.2%). PPVs ranged from 76.3% to 94.3% across the 4 Data Partners. Conclusions The overall PPV of potential AMI cases, as identified using an ICD-9-CM-based algorithm, may be acceptable for safety surveillance; however, PPVs do vary across Data Partners. This validation effort provides a contemporary estimate of the reliability of this algorithm for use in future surveillance efforts conducted using the FDA’s MSDD.

Purpose To describe the Acute Myocardial Infarction (AMI) Validation project, a test case for health outcome validation within the FDA-funded Mini-Sentinel pilot program. Methods The project consisted of four parts: (1) case identification: developing an ICD9-based algorithm to identify hospitalized AMI patients within the Mini-Sentinel Distributed Database; (2) chart retrieval: establishing procedures that ensured patient privacy (collection and transfer of minimum necessary amount of information, and redaction of direct identifiers to validate potential cases of AMI; (3) abstraction and adjudication: trained nurse abstractors gathered key data using a standardized form with cardiologist adjudication; and (4) calculation of the positive predictive value of the constructed algorithm. Results Key decision points included: (1) breadth of the AMI algorithm; (2) centralized vs. distributed abstraction; and (3) approaches to maintaining patient privacy and to obtaining charts for public health purposes. We used an algorithm limited to ICD9 codes 410.x0-410.x1. Centralized data abstraction was performed due to the modest number of charts requested (<155). The project’s public health status accelerated chart retrieval in most instances. Conclusions We have established a process to validate AMI within Mini-Sentinel, which may be used for other health outcomes. Challenges include: (1) ensuring that only minimum necessary data is transmitted by Data Partners for centralized chart review; (2) establishing procedures to maintain data privacy while still allowing for timely access to medical charts; and (3) securing access to charts for public health uses that do not require IRB approval while maintaining patient privacy.

No abstract