Administrative codes combined with medical records based criteria accurately identified bacterial infections among rheumatoid arthritis patients

doi:10.1016/j.jclinepi.2008.06.006

Journal of Clinical Epidemiology

Volume 62, Issue 3, March 2009, Pages 321-327.e7

https://doi.org/10.1016/j.jclinepi.2008.06.006 Get rights and content

Abstract

Objective

To evaluate diagnostic properties of International Classification of Diseases, Version 9 (ICD-9) diagnosis codes and infection criteria to identify bacterial infections among rheumatoid arthritis (RA) patients.

Study Design and Setting

We performed a cross-sectional study of RA patients with and without ICD-9 codes for bacterial infections. Sixteen bacterial infection criteria were developed. Diagnostic properties of comprehensive and restrictive sets of ICD-9 codes and the infection criteria were tested against an adjudicated review of medical records.

Results

Records on 162 RA patients with and 50 without purported bacterial infections were reviewed. Positive and negative predictive values of ICD-9 codes ranged from 54%–85% and 84%–100%, respectively. Positive predictive values of the medical records based criteria were 84% and 89% for “definite” and “definite or empirically treated” infections, respectively. Positive predictive value of infection criteria increased by 50% as disease prevalence increased using ICD-9 codes to enhance infection likelihood.

Conclusion

ICD-9 codes alone may misclassify bacterial infections in hospitalized RA patients. Misclassification varies with the specificity of the codes used and strength of evidence required to confirm infections. Combining ICD-9 codes with infection criteria identified infections with greatest accuracy. Novel infection criteria may limit the requirement to review medical records.

Introduction

Bacterial infections in rheumatoid arthritis (RA) are common [1], [2] and are of growing interest based on an increasing number of serious infections reported in patients receiving biologic therapies [3], [4], [5], [6]. A comprehensive understanding of the associations between infection with RA and the use of specific therapeutic agents has been limited by the absence of objective criteria to correctly identify infection in studies of large populations. Misclassifying infections may mask the risks related to use of particular arthritis medications [6] or could introduce bias if outcome assessment is subjective and reviewers are not blinded to medication exposure. A validated set of diagnostic criteria for a broad range of infections has been lacking in the medical literature and exists mainly for isolated infections such as the Duke criteria for endocarditis [7]. Additionally, although the accuracy of administrative claims data has been studied for various conditions [8], [9], [10], their ability to accurately identify bacterial infections in a hospitalized RA population is largely unknown.

To address these methodological gaps, we sought to evaluate the accuracy of the International Classification of Diseases, Version 9, Clinical Modification (ICD-9) codes commonly used to identify infection outcomes in epidemiologic research, by evaluating a population of RA patients who was hospitalized. Additionally, we constructed medical records based infection criteria for bacterial infections that could be used to validate the presence of infection when applied to abstracted medical information. Both the claims-based algorithms and the medical records based infection criteria were validated against a standard of physician panel review of medical records for hospitalized RA patients.

Section snippets

Study population and infection case identification

After local institutional review board approval, we used the administrative claims data from the University of Alabama at Birmingham health system to identify adults (age ≥18 years at the time of hospitalization) with one or more diagnostic codes for RA (ICD-9, 714.X in any position on the hospital discharge claim), who were hospitalized at our institution between January 2002 and December 2003. We compiled two sets of ICD-9 codes for infections (Appendix A) based on expert consensus (1) a

Results

Of the total 557 RA patients observed in the study period, we abstracted 100% of the relevant records of 162 RA patients hospitalized for the first time with a purported bacterial infection (based on ICD-9 codes) and 50 RA admitted patients without claims for infection (Fig. 1). The mean ± standard deviation age of the patients was 63 ± 14 years and 73% were women. Using claims data, a median of 1.0 bacterial infection was identified per patient per hospitalization; the range of the number of

Discussion

Our study evaluated the diagnostic properties of ICD-9 codes to identify putative bacterial infections in hospitalized RA patients. We found that using ICD-9 codes alone to identify bacterial infections in hospitalized RA patients may misclassify 15%–46% of the infections, depending on the set of codes and the strength of the evidence desired to identify infections. We also developed diagnostic criteria for bacterial infections based on medical records abstraction and showed that they had very

Acknowledgments

We thank Drs. Nenad Avramovski¹ and Ari Robicsek² M.D. for guiding in constructing the medical records based infection criteria, Dr. Robert Lopez M.D. ¹ for guiding in interpreting the radiological findings, Dr. Charles Elson M.D. ¹ for his expertise in gastroe

References (23)

D.T. Durack et al.
New criteria for diagnosis of infective endocarditis: utilization of specific echocardiographic findings. Duke Endocarditis Service
Am J Med
(1994)
Y. Kiyota et al.
Accuracy of Medicare claims-based diagnosis of acute myocardial infarction: estimating positive predictive value on the basis of review of hospital records
Am Heart J
(2004)
S. Schneeweiss et al.
Veteran's affairs hospital discharge databases coded serious bacterial infections accurately
J Clin Epidemiol
(2007)
S. Schneeweiss et al.
A review of uses of health care utilization databases for epidemiologic research on therapeutics
J Clin Epidemiol
(2005)
S.G. Leveille et al.
A new method for identifying antibiotic-treated infections using automated pharmacy records
J Clin Epidemiol
(2000)
J. Baum
Infection in rheumatoid arthritis
Arthritis Rheum
(1971)
M.F. Doran et al.
Frequency of infection in patients with rheumatoid arthritis compared with controls: a population-based study
Arthritis Rheum
(2002)
C.D. Hamilton
Infectious complications of treatment with biologic agents
Curr Opin Rheumatol
(2004)
J. Keane et al.
Tuberculosis associated with infliximab, a tumor necrosis factor alpha-neutralizing agent
New Engl J Med
(2001)
J. Listing et al.
Infections in patients with rheumatoid arthritis treated with biologic agents
Arthritis Rheum
(2005)

J.R. Curtis et al.

Risk of serious bacterial infections among rheumatoid arthritis patients exposed to tumor necrosis factor alpha antagonists

Arthritis Rheum

(2007)

Cited by (68)

Effectiveness and Safety of Biologic Therapy in Hispanic Vs Non-Hispanic Patients With Inflammatory Bowel Diseases: A CA-IBD Cohort Study
2023, Clinical Gastroenterology and Hepatology
There are limited data on outcomes of biologic therapy in Hispanic patients with inflammatory bowel diseases (IBDs). We compared risk of hospitalization, surgery, and serious infections in Hispanic vs non-Hispanic patients with IBD in a multicenter, electronic health record–based cohort of biologic-treated patients.
We identified adult patients with IBD who were new users of biologic agents (tumor necrosis factor α [TNF-α] antagonists, ustekinumab, vedolizumab) from 5 academic institutions in California between 2010 and 2017. We compared the risk of all-cause hospitalization, IBD-related surgery, and serious infections in Hispanic vs non-Hispanic patients using 1:4 propensity score matching and survival analysis.
We compared 240 Hispanic patients (53% male; 45% with ulcerative colitis; 73% TNF-α antagonist–treated; 20% with prior biologic exposure) with 960 non-Hispanic patients (51% male; 44% with ulcerative colitis; 67% TNF-α antagonist–treated; 27% with prior biologic exposure). After propensity score matching, Hispanic patients were younger (37 ± 15 vs 40 ± 16 y; P = .02) and had a higher burden of comorbidities (Elixhauser index, >0; 37% vs 26%; P < .01), without any differences in patterns of medication use, burden of inflammation, and hospitalizations. Within 1 year of biologic initiation, Hispanic patients had higher rates of hospitalizations (31% vs 23%; adjusted hazard ratio [aHR], 1.32; 95% CI, 1.01–1.74) and IBD-related surgery (7.1% vs 4.6%; aHR, 2.00; 95% CI, 1.07–3.72), with a trend toward higher risk of serious infections (8.8% vs 4.9%; aHR, 1.74; 95% CI, 0.99–3.05).
In a multicenter, propensity score–matched cohort of biologic-treated patients with IBD, Hispanic patients experienced higher rates of hospitalization, surgery, and serious infections. Future studies are needed to investigate the biological, social, and environmental drivers of these differences.
The Association Between Proton Pump Inhibitor Exposure and Key Liver-Related Outcomes in Patients With Cirrhosis: A Veterans Affairs Cohort Study
2022, Gastroenterology
The impact of proton pump inhibitory (PPI) medications on adverse outcomes in cirrhosis remains controversial. We aimed to evaluate the association between PPI exposure and all-cause mortality, infection, and decompensation in a large national cohort.
This was a retrospective study of patients with cirrhosis in the Veterans Health Administration. PPI exposure was classified as a time-updating variable from the index time of the cirrhosis diagnosis. Inverse probability treatment weighting–adjusted Cox regression was performed with additional adjustment for key time-varying covariates, including cardiovascular comorbidities, gastrointestinal bleeding (GIB), and statin exposure.
The study included 76,251 patients, 23,628 of whom were on a PPI at baseline. In adjusted models, binary (yes/no) PPI exposure was associated with reduced hazard of all-cause mortality in patients with hospitalization for GIB (hazard ratio [HR], 0.88; 95% confidence interval [CI], 0.84–0.91; P < .001) but had no significant association in all others (HR, 0.99; 95% CI, 0.97–1.02; P = .58). However, cumulative PPI exposure was associated with increased mortality in patients without hospitalization for GIB (HR, 1.07 per 320 mg-months [omeprazole equivalents]; 95% CI, 1.06–1.08; P < .001). PPI exposure was significantly associated with severe infection (HR, 1.21; 95% CI, 1.18–1.24; P < .001) and decompensation (HR, 1.64; 95% CI, 1.61–1.68; P < .001). In a cause-specific mortality analysis, PPI exposure was associated with increased liver-related mortality (HR, 1.23; 95% CI, 1.19–1.28) but with decreased nonliver-related mortality (HR, 0.88; 95% CI, 0.85–0.91).
PPI exposure is associated with increased risk of infection and decompensation in cirrhosis, which may mediate liver-related mortality. However, PPI use was associated with reduced all-cause mortality in those with prior GIB, suggesting benefit in the presence of an appropriate indication.
Leveraging electronic health record data for clinical trial planning by assessing eligibility criteria's impact on patient count and safety
2022, Journal of Biomedical Informatics
To present an approach on using electronic health record (EHR) data that assesses how different eligibility criteria, either individually or in combination, can impact patient count and safety (exemplified by all-cause hospitalization risk) and further assist with criteria selection for prospective clinical trials.
Trials in three disease domains – relapsed/refractory (r/r) lymphoma/leukemia; hepatitis C virus (HCV); stages 3 and 4 chronic kidney disease (CKD) – were analyzed as case studies for this approach. For each disease domain, criteria were identified and all criteria combinations were used to create EHR cohorts. Per combination, two values were derived: (1) number of eligible patients meeting the selected criteria; (2) hospitalization risk, measured as the hazard ratio between those that qualified and those that did not. From these values, k-means clustering was applied to derive which criteria combinations maximized patient counts but minimized hospitalization risk.
Criteria combinations that reduced hospitalization risk without substantial reductions on patient counts were as follows: for r/r lymphoma/leukemia (23 trials; 9 criteria; 623 patients), applying no infection and adequate absolute neutrophil count while forgoing no prior malignancy; for HCV (15; 7; 751), applying no human immunodeficiency virus and no hepatocellular carcinoma while forgoing no decompensated liver disease/cirrhosis; for CKD (10; 9; 23893), applying no congestive heart failure.
Within each disease domain, the more drastic effects were generally driven by a few criteria. Similar criteria across different disease domains introduce different changes. Although results are contingent on the trial sample and the EHR data used, this approach demonstrates how EHR data can inform the impact on safety and available patients when exploring different criteria combinations for designing clinical trials.
Comparative Risk of Serious Infections With Tumor Necrosis Factor α Antagonists vs Vedolizumab in Patients With Inflammatory Bowel Diseases
2022, Clinical Gastroenterology and Hepatology
Citation Excerpt :
Ideally, infections would be adjudicated by medical record review and microbiology data, but this level of data is unavailable in claims databases. However, our definition of serious infections requiring hospitalization has been validated with a high positive predictive value.17,18 In summary, in a large administrative claims database study of approximately 6000 patients with IBD between 2014 and 2018, we observed that vedolizumab is associated with lower risk of serious infections as compared with TNF-α antagonists in patients with UC but not in patients with CD.
We conducted a retrospective cohort study comparing the risk of serious infections between patients treated with tumor necrosis factor-a (TNFa) antagonists vs. vedolizumab in patients with inflammatory bowel diseases (IBD).
Using an administrative claims database, we identified patients with IBD who were new-users of either TNFa antagonists or vedolizumab between 2014-2018 and had insurance coverage for at least 1y before and after treatment initiation. We compared the risk of serious infections (infections requiring hospitalization) between patients treated with vedolizumab or TNFa antagonists using marginal structural Cox proportional hazard models adjusted for baseline disease characteristics, healthcare utilization, comorbidities, and time-varying use of corticosteroids, immunomodulators and opiates.
We included 4881 patients treated with TNFa antagonists (age, 41 ± 15y, 60% with Crohn’s disease [CD]) of whom 434 developed serious infections over 5786 person-year [PY] follow-up, and 1106 patients treated with vedolizumab (age, 44 ± 16y, 39% with CD) of whom 86 developed serious infections over 1040-PY follow-up. Vedolizumab was associated with 46% lower risk of serious infections as compared with TNFa antagonists in patients with ulcerative colitis (HR,0.54 [95% CI,0.35-0.83), but no significant differences were observed in patients with CD (HR,1.30 [0.80-2.11]). Vedolizumab was associated with lower risk of extra-intestinal serious infections in patients with UC, but higher risk of gastrointestinal serious infections in patients with CD.
In an observational study of patients with IBD, vedolizumab was associated with lower risk of serious infections as compared with TNFa antagonists, in patients with UC, but not in patients with CD.
Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method
2021, Journal of Biomedical Informatics
Citation Excerpt :
Appendicitis trial: The index event was diagnosis for acute uncomplicated appendicitis.[55,56] Exclusion criteria included age less than 18 years old; severe sepsis or septic shock;[57] conditions with altered immune response;[56,58] evidence of immunodeficiency;[46,50–52] uncompensated liver failure;[59] medication usage for active inflammatory bowel disease (IBD);[60] malignancy;[54] evidence of pregnancy;[53] concurrent hemodialysis, peritoneal dialysis, or treatment using indwelling venous catheters; placement of surgical device (e.g., pacemaker, joint prosthesis, mechanical valve); evidence of indwelling left ventricular assist device (LVAD); infection that requires treatment with another antibiotic;[61] prior enrollment in study or receiving other investigational agent; and prior abdominal/pelvic surgery. Descriptive statistics are presented for TP and PE, as per EHR clinical characteristics extracted by the OHDSI FeatureExtraction R package.[62]
To present a generalizability assessment method that compares baseline clinical characteristics of trial participants (TP) to potentially eligible (PE) patients as presented in their electronic health record (EHR) data while controlling for clinical setting and recruitment period.
For each clinical trial, a clinical event was defined to identify patients of interest using available EHR data from one clinical setting during the trial’s recruitment timeframe. The trial’s eligibility criteria were then applied and patients were separated into two mutually exclusive groups: (1) TP, which were patients that participated in the trial per trial enrollment data; (2) PE, the remaining patients. The primary outcome was standardized differences in clinical characteristics between TP and PE per trial. A standardized difference was considered prominent if its absolute value was greater than or equal to 0.1. The secondary outcome was the difference in mean propensity scores (PS) between TP and PE per trial, in which the PS represented prediction for a patient to be in the trial. Three diverse trials were selected for illustration: one focused on hepatitis C virus (HCV) patients receiving a liver transplantation; one focused on leukemia patients and lymphoma patients; and one focused on appendicitis patients.
For the HCV trial, 43 TP and 83 PE were found, with 61 characteristics evaluated. Prominent differences were found among 69% of characteristics, with a mean PS difference of 0.13. For the leukemia/lymphoma trial, 23 TP and 23 PE were found, with 39 characteristics evaluated. Prominent differences were found among 82% of characteristics, with a mean PS difference of 0.76. For the appendicitis trial, 123 TP and 242 PE were found, with 52 characteristics evaluated. Prominent differences were found among 52% of characteristics, with a mean PS difference of 0.15.
Differences in clinical characteristics were observed between TP and PE among all three trials. In two of the three trials, not all of the differences necessarily compromised trial generalizability and subsets of PE could be considered similar to their corresponding TP. In the remaining trial, lack of generalizability appeared present, but may be a result of other factors such as small sample size or site recruitment strategy. These inconsistent findings suggest eligibility criteria alone are sometimes insufficient in defining a target group to generalize to. With caveats in limited scalability, EHR data quality, and lack of patient perspective on trial participation, this generalizability assessment method that incorporates control for temporality and clinical setting promise to better pinpoint clinical patterns and trial considerations.
Special considerations on interventions: Biologics
2021, Pragmatic Randomized Clinical Trials: Using Primary Data Collection and Electronic Health Records
This chapter will provide an overview of pragmatic trials in the biologic medicines setting. This chapter will start with an introduction to biologic medicines in terms of how they are defined, how they are broadly used in clinical medicine, and types of research questions typically studied in biologics using a pragmatic clinical trial design. A high-level summary of general pharmacovigilance concerns, such as immunogenicity, manufacturing variability, and identification/traceability is provided. Biologic-specific considerations will include topics such as the complexity of patient profiles and treatment. A biologic focus of pragmatic trial designs and their applicability to biologics, common endpoints, and strengths and limitations of real world data sources for pragmatic trials will be described. This chapter will be valuable to researchers and scientists who are interested in understanding considerations for conducting a pragmatic clinical trial of biologic medicines.

View all citing articles on Scopus

View full text

Original ArticleAdministrative codes combined with medical records based criteria accurately identified bacterial infections among rheumatoid arthritis patients

Abstract

Objective

Study Design and Setting

Results

Conclusion

Introduction

Section snippets

Study population and infection case identification

Results

Discussion

Acknowledgments

Am J Med

Am Heart J

J Clin Epidemiol

J Clin Epidemiol

J Clin Epidemiol

Infection in rheumatoid arthritis

Arthritis Rheum

Frequency of infection in patients with rheumatoid arthritis compared with controls: a population-based study

Arthritis Rheum

Infectious complications of treatment with biologic agents

Curr Opin Rheumatol

Tuberculosis associated with infliximab, a tumor necrosis factor alpha-neutralizing agent

New Engl J Med

Infections in patients with rheumatoid arthritis treated with biologic agents

Arthritis Rheum

Risk of serious bacterial infections among rheumatoid arthritis patients exposed to tumor necrosis factor alpha antagonists

Arthritis Rheum

Original Article
Administrative codes combined with medical records based criteria accurately identified bacterial infections among rheumatoid arthritis patients