Article Text
PDF
Abstract
Objectives Although process measures to assess quality of care in systemic lupus erythematosus (SLE) are available, their relationship to long-term outcomes has not been studied. Using a prospective, longitudinal cohort study, we examined the associations between high-quality care and two important SLE outcomes, disease activity and damage.
Methods Data were derived from the University of California, San Francisco Lupus Outcomes Study. Participants were followed from 2009 through 2013, responding to yearly surveys. Primary outcomes in this study were clinically meaningful increases in disease activity and damage, assessed by the Systemic Lupus Activity Questionnaire (SLAQ) and the Brief Index of Lupus Damage (BILD), respectively. Using multivariable regression, we examined the relationship between high performance on 13 validated quality measures (receipt of ≥85% of quality measures), and disease outcomes, adjusting for disease status, sociodemographic characteristics, healthcare services and follow-up time.
Results The 737 participants were eligible for a mean of five quality measures (SD 2, range 2–12). There were 155 and 162 participants who had clinically meaningful increases in SLAQ and BILD, respectively. In our models, we found no statistically significant relationship between performance on quality measures and changes in SLAQ. However, receiving higher-quality SLE care was significantly protective against increased disease damage (adjusted OR 0.4, 95% CI 0.4 to 0.7), even after adjusting for covariates.
Discussion In this community-based cohort, we illustrate for the first time a strong link between processes of care, defined by SLE quality measures, and the subsequent accumulation of disease damage, an important outcome.
- Health Services Research
- Quality Measurement
- Chronic Disease Management
- Ambulatory Care
Statistics from Altmetric.com
Establishing the relationship between process quality measures and outcomes is a crucial step in efforts to improve quality. In systemic lupus erythematosus (SLE), a severe autoimmune disease primarily affecting women, available quality measures have focused on processes of care, based on the assumption that processes that are beneficial in clinical studies or that are agreed upon by experts will lead to improvements in patient outcomes.1 ,2 However, there has been no research regarding the effectiveness of these processes in real-world settings. This information is critical to understanding the potential impact of creating incentives to improve process quality in SLE in clinical practice.
Although Donabedian conceptualised processes of care as part of a continuum that leads to better outcomes,3 some have argued that it may be more fruitful to directly measure outcomes when evaluating quality. This approach has intuitive appeal since outcomes are what matter most to patients and improved outcomes are the ultimate goal of a high-functioning health system. However, there remain significant challenges to this approach. In chronic diseases such as SLE, key outcomes such as accumulated organ damage may take years to develop and are therefore perceived as not entirely within the immediate control of individual providers. In addition, risk adjustment of averaged patient outcomes within a clinic or healthcare system is daunting in a disease that can affect virtually any organ in the body and has dramatically different levels of severity in the population. Until these and other methodological issues are addressed through research, process measures have appeal in that they can more immediately inform quality improvement activities. Furthermore, a growing number of studies have reported deficits in process quality in SLE as well as disparities in care.4–7 Establishing a link between these process measures and outcomes is important in further assessing their validity and usefulness for quality improvement, as well as for narrowing the striking health disparities in the condition.
The aim of this study was to examine the extent to which higher performance on SLE process measures that have evidence-based links to outcomes predicted actual changes in health outcomes over time. We studied individuals with SLE enrolled in a prospective, longitudinal study, in which a set of validated quality measures has been systematically collected over a 4-year period. We focused on changes in two critical disease outcomes, SLE disease activity and permanent disease damage.
Methods
Data source
Data were derived from the University of California, San Francisco (UCSF) Lupus Outcomes Study (LOS), a large observational cohort of English-speaking individuals with SLE, recruited in several settings, including academic rheumatology offices, community rheumatology offices and non-clinical sources. All SLE diagnoses were confirmed by a formal medical record review to document American College of Rheumatology criteria for SLE.8 Participants completed a structured telephone interview each year conducted by trained survey workers covering topics such as sociodemographics, disease status, medications, healthcare use and health insurance coverage using validated survey items.9 Recruitment for the LOS began in 2003; the present study incorporates data from 878 LOS participants interviewed between February 2009 and March 2013. A majority of respondents (71%) resided in California at the time of the first interview; the remainder resided in 36 other states.
The UCSF Committee on Human Research approved the study protocol, and all participants gave written informed consent.
Outcomes
The primary outcomes in this study were changes in SLE disease activity and accumulated disease damage. For disease activity, we used the Systemic Lupus Activity Questionnaire (SLAQ) (range 0–47), a patient self-report measure of disease activity across several domains contained in its physician-assessed counterpart, the Systemic Lupus Activity Measure (SLAM).10 The SLAQ has been shown to have adequate reliability, construct validity and responsiveness.10 ,11 The SLAQ is collected yearly in the LOS survey. For disease damage, we used the Brief Index of Lupus Damage (BILD) (range 0–26), which relies on patient's self-report of disease damage across several domains contained in the physician-assessed SLICC/ACR Damage Index (SDI), and has been shown to have good content, criterion and construct validity.12 ,13 The BILD was collected for the first time in the 2007–2008 interviews and again in the 2012–2013 interviews.
Because cutpoints for meaningful changes in disease activity and damage have not been established for either the SLAQ or the BILD, we relied on information from psychometric validation studies. For the SLAQ, we relied on a commonly used psychometric definition of a clinically meaningful increase of ≥0.5 SDs, which in a previously study of the SLAQ has been found to correlate with worse physical functioning, global health ratings and decreased ability to perform valued life activities11 ,14; this corresponded to an increase of ≥ 4 points between the first and fourth year of observation. For BILD, we defined meaningful change as an increase of ≥2 points, corresponding to the top quartile of change in damage scores, an increase that has been prospectively found to be associated with a sixfold increase in mortality as well as higher hospitalisation rates.15
Unlike damage items assessed in the BILD, which generally accumulate slowly over a period of years, and therefore would not be expected to have large year-to-year changes, disease activity can change quickly in SLE. We therefore examined changes in SLAQ scores from year-to-year during the study period, allowing multiple observations per person. In addition, because a disease flare might occur at any time during the study period, we compared those who had an increase in SLAQ of ≥4 points over baseline in any year compared with those who did not.
Principal independent variable: process quality measures
Our primary independent variable was performance on 13 previously validated SLE quality measures. These process measures were derived from the SLE Quality Indicator (QI) Project and were developed using a validated method combining systematic literature reviews and expert panel ratings.1 ,4 In that project, potential QIs were extracted from a systematic literature review of clinical practice guidelines pertaining to SLE. An advisory panel revised and augmented these candidate indicators. A modification of the RAND/UCLA Appropriateness Method was then employed; systematic literature reviews were performed for each QI, linking the proposed processes of care to potential improved health outcomes and after reviewing this evidence, a second interdisciplinary expert panel convened to discuss the evidence and provide final ratings on the validity and feasibility of each QI. Twenty QIs were rated as valid and feasible, and 13 of these were later operationalised for patient self-report. We created detailed interview algorithms for those QIs and assessed their validity. The technical details of measure specifications and validation have been published elsewhere.4
For the analysis, we defined a pass rate as the proportion of eligible recommended services received. Participants were eligible for a mean of five services (range 2–11). We considered a pass rate of ≥85% (approximately corresponding to the top quintile of performance) indicative of high-quality care. This cutpoint was chosen since we were specifically interested in understanding the impact of receiving high-quality care.
The quality measures were collected in each interview beginning in 2009, which we call the baseline year for the present analysis. For the main analysis of change in disease outcomes over 4 years, we used the baseline pass rate. For the analysis of year-to-year change in SLAQ, we analysed the change in SLAQ as a function of the pass rate in the first year of the pair.
In addition to looking at an overall pass rate, we grouped the quality measures into several categories (based on the original development of the quality measures): general preventive strategies, osteoporosis prevention and treatment, drug toxicity prevention and monitoring, renal disease prevention and treatment, and monitoring for cardiovascular risk factors.1 Participants were considered to have ‘passed’ that group if they had received all services for which they were eligible; those who were not eligible for any of the services in that group were excluded. For the five quality measures for which a large proportion of participants were eligible (≥60%), we also examined how these individual measures predicted increased disease damage and/or activity. The remaining quality measures were applicable to fewer than 25% of participants.
Covariates
We considered a number of other potential predictors of SLE outcomes. Sociodemographic variables, all assessed during the baseline year, included age, sex, race/ethnicity (white and non-white or Hispanic), disease duration, educational attainment and poverty (household income ≤125% of the Federal poverty limit). Measures of healthcare use included total number of physician visits reported in the prior year and number of quality measures for which an individual was eligible. We included these measures as we anticipated that those with higher levels of use might have more opportunities to receive care consistent with the quality measures. The presence or absence of health insurance as well as variables for the source (public vs private payor) and type (managed care vs fee for service) of insurance were also included in the analyses. Finally, we examined the types of physicians seen by participants, categorising individuals based on whether they saw a generalist and/or a rheumatologist.
Study sample
Our primary analysis focused on the change in SLE activity and damage from the beginning of the study period to the end (2009–2013). Most survey participants were enrolled prior to 2009, and their data were included from that point forward. A small subset (8%) did not join the study until 2010 and were followed for only 3 years. Follow-up rates are extremely high in the LOS, averaging 94% per year; this figure approaches 96% after excluding the deceased from the denominator.
Item non-response in the study is very low; we excluded only six participants who did not have a baseline BILD score, leaving a final sample of 878. In the analysis of year-to-year change in SLAQ, all pairs of interviews were included, up to three per person, for a total of 2251 observations.
Statistical analyses
For the primary analysis, we used logistic regression to model a clinically meaningful change in SLAQ or BILD, defined above, as a function of high-quality care (receiving ≥85% of eligible services), controlling for the sociodemographics, health use, physician access and health insurance variables described above, as well as for baseline levels of outcome measures and the amount of time elapsed from baseline to follow-up. For annual change in SLAQ, we accounted for the multiple observations per person through use of cluster-correlated robust variance estimation methods.16 We also used these same models to separately investigate categories of quality measures and the more common individual measures.
We were concerned that greater disease activity or severity would lead to higher use and higher numbers of eligible services, and that the more frequent contact with the healthcare system would lead to a situation in which higher pass rates were associated with increased damage or disease activity. Therefore, we undertook several sensitivity analyses to try to uncover these more complex relationships. We examined interaction terms for pass rate and number of eligible services, and we also conducted analyses in which we stratified the dataset according to the number of eligible services. (2–3, 4–6, 7–11) Since none of the results varied from the main findings, we present only the simpler models. Moreover, we investigated interactions between pass rate and each of the other covariates but found no significant effect modification for any of them.
Because there was no a priori pass rate to use as the cutpoint for high-quality SLE care, we looked at alternate values on either side of the 85% pass rate in sensitivity analyses, ≥80% and ≥90% pass rates. We also examined those receiving lower quality care, which we defined as receiving <50% of eligible services, a definition that encompassed 19% of the sample.
Finally, we considered that performance on quality measures could change at each interview and therefore analysed the pass rates on a longitudinal basis. The overall average pass rate did not change across the three interview waves, but there was considerable within-person variability, with only moderate year-to-year correlations (r=0.4). We therefore grouped participants as always, sometimes or never having reached the cutpoint of 85% pass rate over the 3 years.
Analyses were conducted in SAS V.9.2 (Cary, North Carolina, USA) and STATA V.12 (College Station, Texas, USA).
Results
Of 878 participants in the LOS who completed interviews in 2009 or 2010, 737 (84%) were interviewed in 2012 (table 1). These individuals comprise the long-term follow-up sample for this study. Participants were mostly female, 62% non-Hispanic whites, with a mean age of 50±13. As a group, they were well educated, but 15% had poverty-level incomes. They had long-standing disease, were mostly insured and were very likely to have seen a rheumatologist and a generalist in the past year. Sixteen per cent of individuals had used moderate doses of glucocorticoids over the last year, defined as (≥7.5 mg for ≥ 90 days) and 29% met American College of Rheumatology criteria for lupus nephritis at enrolment.8
Characteristics of Lupus Outcomes Study sample in baseline year, by follow-up status in 2012
The overall quality measure pass rate at baseline was 64% (SD 22%); 18% of individuals had a pass rate ≥85%, our primary outcome.
The 141 individuals who were not followed to 2012 did not differ from those who were followed by gender, disease duration, race/ethnicity, income, insurance source or physician access. However, participants in the group that was not followed were older, less likely to have a bachelor’s degree or to be enrolled in a health maintenance organisation (HMO). They had more contact with the healthcare system in the year before their baseline visit, were eligible for more services covered by the quality measures, had more glucocorticoid use and fewer renal met criteria for lupus nephritis. Thirty-six (26%) of these individuals had died in the intervening years.
The sample for the year-to-year analysis consisted of the 828 (94%) participants who were followed for at least 1 year beyond the baseline interview.
At baseline, participants had a mean SLAQ of 11.6±7.9 (range 0–46). Among those followed through 2012, 21% (n=155) had a significant increase in SLAQ of ≥4 points (≥0.5 cohort SD). In the year-to-year analysis, 18% of observations had a change of that magnitude in single year (table 2). The pass rate was not associated with either year-to-year or a long-term increase in disease activity, with or without adjustment for covariates. Adjustment resulted in minimal changes in effect sizes. In multivariable models, higher baseline BILD, lower baseline SLAQ and lower levels of education were associated with year-over-year and long-term increases in SLAQ.
Relationship between lupus disease activity and quality of care, with and without adjustment for disease status, health services and sociodemographic characteristics
Baseline BILD scores averaged 2.1±2.1; 22% (n=162) of participants had an increase of ≥2 points the next time they completed the BILD, in 2012. In contrast to the results for disease activity, a pass rate ≥85% was strongly protective against a clinically meaningful increase in disease damage (OR 0.4, 95% CI 0.2 to 0.7), controlling for other covariates (table 3). Eligibility for ≥7 quality measures, more healthcare use and higher baseline BILD and SLAQ scores were predictive of increased disease damage.
Risk of increased disease damage associated with receiving appropriate care for SLE, with and without adjustment for disease status, health services and sociodemographic characteristics
In our analysis looking at whether individual quality measures were responsible for the protective effect of higher-quality SLE care on disease damage, we found that none of the individual quality measures was associated with a change in BILD score. In addition, no category of services had a statistically significant protective effect on damage (these results can be found in the online supplementary appendix).
As sensitivity analyses, we examined the multivariable model for increased BILD using alternate cutpoints for the pass rate, ≥90%, ≥80% and <50%. The ≥90% pass rate, capturing 15% of the sample, was slightly more protective against increased damage than the ≥85% cutpoint (OR 0.35, 95% CI 0.18 to 0.65). The ≥80% pass rate, encompassing 29% of participants, was somewhat less protective against increased damage (OR 0.65, 95% CI 0.41 to 1.03). By contrast, there was no effect on cumulative damage for the low end of the quality measure pass rates (<50%; OR 0.90, 95% CI 0.52 to 1.56).
Because there was significant within-person variation in the pass rates over the 3 years of observation, we also devised a cumulative measure of the pass rate, in which individuals always, sometimes or never reached the cutpoint of a 85% pass rate. In these analyses, the relationship between the process of care and outcomes was analogous to our primary analyses.
Participants who met the 85% pass rate in at least 1 year were less likely to have increased damage compared with those who never met the cutpoint (OR 0.6, 95% CI 0.40 to 0.95). As in the primary analysis, there was no effect of quality of care on disease activity.
Discussion
In this study, we evaluated whether higher quality of care for SLE, as assessed by performance on process measures, was associated with improved patient outcomes over time. We found a strong protective effect of higher quality of care on the accumulation of organ damage, but no effect on disease activity. The process–outcome relationship for disease damage remained even after adjusting for a wide range of sociodemographic, disease-related and healthcare use factors that potentially impact outcomes in SLE. These findings support the notion that high process quality is associated with an important long-term disease outcome and contributes to our thinking about the usefulness of process measures for quality improvement in SLE.
There are several possible explanations for the protective effect of higher quality of care on disease damage but not disease activity. Process measures in the SLE Quality Indicators Project addressed aspects of care that are important in the prevention of disease damage, such as cardiovascular risk screening, osteoporosis screening and treatment, establishing a glucocorticoid management plan (since glucocorticoids are a major contributor to accumulated damage), and drug toxicity prevention and monitoring.1 The relatively high burden of organ damage resulting from these comorbidities as well as drug toxicity in SLE may have allowed us to see a stronger than expected process–outcome link. Indeed, in other chronic conditions, it has been difficult to establish an association between higher process quality and health outcomes, although some studies have reported modest associations.17–21 For disease activity, however, we did not observe this same process–outcome relationship. Fewer of the SLE QIs addressed management strategies for controlling disease activity, in large part because the heterogeneity of the disease makes it challenging to create measures that apply uniformly across the patient population. It is therefore possible that disease activity in SLE may be primarily driven by processes inadequately captured by the measures studied. This, coupled with the relatively stable disease activity in many LOS participants for the quality measure domains assessed, such as renal disease, may have limited our ability to detect a process–outcome link.
While we found a relatively large protective effect of high-quality care on disease damage accumulation, the effect appears unrelated to any individual or category of SLE process measures. It is possible that the process quality measures studied here may be markers for larger elements of healthcare quality that are unmeasured; the SLE QIs may encompass only part of the pathways of care that contribute to improved outcomes, and higher performance on these measures as a group may be a proxy for better care overall that results in decreased damage. For example, factors such as timely access to care, staffing, organisational culture and systems of care, all unmeasured here, may affect process quality and health outcomes. In addition, it is possible that the impact of high-quality care may become apparent only when care is of consistently high quality. Further research and larger studies are needed to elucidate these relationships.
Prior work has demonstrated both substantial gaps in quality on process measures in SLE and also disparities in quality and outcomes. Using data from the LOS, we found that individuals with SLE received approximately 65% of services recommended in SLE process measures.4 Studies examining osteoporosis prevention and treatment, cardiovascular risk screening, reproductive healthcare, immunisations and also provision of recommended SLE therapies have identified gaps in care for each of these areas.5–7 ,22 ,23 Patients who have historically had a higher risk for poor outcomes in SLE, including individuals with low socioeconomic status, were more likely to receive lower quality of care across many of these studies, an effect that seems at least partially mediated by health system characteristics, such as insurance or the systems of care.4 Insofar as we demonstrate a strong protective effect of higher process quality on a key health outcome in SLE, our study suggests that efforts to measure and improve process quality should be evaluated as a potential tool to reduce the striking disparities that are well documented in the condition.24
The strength of our study includes the prospective collection of data on processes of care and validated disease outcomes over time in a community-based cohort of individuals in SLE. This allowed us to perform the first study examining the real-world relationship between processes of care and SLE outcomes. However, there are also limitations to our study that are important to consider. First, the findings of our study may not be generalisable to all SLE patients in the USA since a majority of LOS participants reside in California and almost two-thirds are white. Second, the outcome measures used in this study are self-reported, which introduces some imprecision compared with analogous clinical assessments. However, both the SLAQ and the BILD have been validated and found to have good content validity and construct validity, including recent research that demonstrates that higher BILD scores are strongly related to mortality.10–12 ,15 Nevertheless, future studies using physician-assessed disease activity and damage indices are warranted to further validate our findings.
In addition, although we were able to include a wide array of health, use and other factors in our analysis, we cannot definitively say that the relationship between process of care and disease damage is causal. This is because of the possibility of unmeasured confounding factors. For example, some individuals may have unmeasured characteristics that make them likely to receive higher-quality care and have improved health outcomes. Similarly, high process quality may be related to unmeasured healthcare factors that are primarily driving improved outcomes. Our findings are important in that they suggest that future efforts to define and test randomised interventions to improve process quality in SLE are needed.
Our results have important implications for healthcare delivery in SLE. First, although our findings add to accumulating evidence that there are significant gaps in quality of care for SLE, they also suggest that very high-quality care is achievable. Second, by demonstrating a strong process–outcome link for the first time in SLE, our results suggest that one plausible mechanism of poor outcomes in SLE is low healthcare quality. This information can be useful in thinking about strategies to improve outcomes and reduce disparities in the condition. Third, given the challenges of developing risk-adjusted and broadly applicable outcome measures in a complex and heterogeneous disease like SLE, our findings suggest that it may be fruitful to focus on improving process quality in the short term until appropriate outcome measures or risk-adjustment strategies are available. Finally, as quality measurement and improvement efforts continue, our study suggests that further developing quality measures and then creating incentives for their use in SLE is justified to improve what ultimately matters to most patients—health outcomes.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Contributors JY, LT, GS, PPK and EHY all conceived of the study design, analysed and interpreted the data, drafted the article and revised it critically for important intellectual content and final approval of the version to be published.
-
Funding Research reported in this manuscript was supported by the National Institute of Arthritis and Musculoskeletal and Skin diseases under award numbers K23 AR060259, P60 AR053308 and 2R01 AR056476. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
-
Competing interests None.
-
Ethics approval University of California, San Francisco.
-
Provenance and peer review Not commissioned; externally peer reviewed.