Article Text
PDF
Abstract
Objective: To assess available evidence on the use of end-points (outcome measures) in clinical trials in systemic lupus erythematosus (SLE), as a part of the development of evidence-based recommendations for points to consider in clinical trials in SLE.
Methods: The European League Against Rheumatism (EULAR) Task Force on SLE comprised 19 specialists, a clinical epidemiologist and a research fellow. Key questions addressing the evidence for clinical trial end-points in SLE were compiled using the Delphi technique. A systematic search of the PubMed and Cochrane Library databases was performed using McMaster/Hedges clinical query strategies and an array of relevant terms. Evidence was categorised based on sample size and type of design, and the categories of available evidence were identified for each recommendation. The strength of recommendation was assessed based on the category of available evidence and agreement on the statements was measured across the 19 specialists.
Results: Eight questions were generated regarding end-points for clinical trials. The evidence to support each proposition was evaluated. The literature review revealed that most outcome measures used in phase 2/3 trials in SLE have not been formally validated in clinical trials, although some indirect validation has been undertaken.
Conclusion: This systematic literature review forms the evidence base considered in the development of the EULAR recommendations for end-points in clinical trials in SLE.
Statistics from Altmetric.com
Systemic lupus erythematosus (SLE) is a complex multisystem autoimmune disease that is most common in women during their reproductive years. Certain ethnic groups (eg, of African origin) are predisposed to develop this disease more frequently than others, and have a worse prognosis in terms of increased risk of developing renal involvement, accumulating damage and of increased mortality.1 2 Currently, antimalarial drugs, corticosteroids, immunosuppressive and cytotoxic agents are the mainstays of therapy. These treatments have many complications even though they have improved symptom control and reduced rates of renal failure. Consequently there is increasing interest in developing more specific therapies with improved efficacy and safety profiles over traditional drug treatments.
There have been previous attempts at defining the type of outcome measures that should be used in clinical studies and trials in SLE, but no formal recommendations have been published. The first initiative was published by the Systemic Lupus International Collaborating Clinics (SLICC) in 1996 and defined the concepts of assessing separately disease activity, chronic damage and quality of life.3 These recommendations were endorsed by the Outcome Measures in Rheumatology (OMERACT) IV conference in 1998, by which time there was more literature to review and comment upon but still relatively few randomised clinical trials (RCTs) of lupus manifestations other than nephritis.4 5 More recently, the US Food and Drug Administration (FDA) has reviewed the literature and identified issues for the pharmaceutical industry to consider when planning clinical trials (http://www.fda.gov/cder/guidance/6496dft.pdf).
The European League Against Rheumatism (EULAR) has developed standard operating procedures for reviewing the evidence base for medical practice.6 This approach ensures a high level of intrinsic quality and comparability, and has now been applied to developing recommendations for clinical practice and clinical trials in rheumatic diseases. We undertook the task of developing recommendations for the management of various aspects of SLE (published separately)7 and points to consider for conducting clinical trials in SLE8 using a combination of research-based evidence and expert consensus. In this paper, we review the evidence for eight key statements about possible end-points for clinical trials in SLE.
METHODS
The expert committee and selection of questions
An expert committee was formed that comprised 19 specialists, 1 clinical epidemiologist (JPAI) and 1 research fellow (GB), representing 12 European countries. EULAR standardised operating procedures were followed for the elaboration and evaluation of the evidence for conducting clinical trials with the aim of devising recommendations for dissemination and implementation in due course. The initial discussion focused on the spectrum of disease to be studied, the patient population and the target audience to receive guidelines concerning clinical trial end-points in SLE. The committee voted to define the focus of the process to be clinical trials of interventions for any aspect of lupus in adult patients with lupus of any ethnic background. The target audience was defined as all practicing doctors concerned with planning or evaluating clinical trials in SLE, epidemiologists and other research methodologists, the pharmaceutical industry and the licensing authorities. Eight questions addressing the evidence for clinical trial methodology in SLE were selected using a modified Delphi technique.
Systematic literature search
A systematic search of the PubMed and Cochrane Library databases was performed, and all English language publications up to January 2006 were considered, as previously described.9 When the searches above failed to identify any pertinent relevant study for a specific question, more sensitive searches were performed using the specific terms/names of the item for which no relevant studies were identified.
Literature screening and categories of evidence
Retrieved items were screened for eligibility based on their title, abstract and/or full content. Animal studies, narrative review articles, commentaries, conference abstracts or statements, expert opinion statements and guidelines were excluded. Since the topics varied widely and the retrieved items were heterogeneous in many methodological aspects, no systematic scoring system was used. Evidence was categorised according to study design using a traditional rating scale and the strength of the evidence was graded combining information on the design and validity of the available data.7
Expert opinion approach and strength of statements
The results of the literature search were summarised, aggregated and distributed to the expert committee. This formed the basis for discussion during a second meeting of the Task Force and resulted in agreement on eight final statements about end-points for clinical trials in SLE. The strength of the statements was graded A–D and ratified by the expert committee. This was followed by a consensus building process to derive a set of recommendations for points to consider when conducting clinical trials in SLE, based on the limited currently available literature combined with expert opinion. Members of the Task Force were also asked to state their strength of agreement with each statement in the tables, to the best of their ability and experience on a scale of 1–10 (10 being full agreement and 1 being total disagreement).
RESULTS
(See also table 1.)
Have any, and if so which, intermediate outcome measures (including activity indices and predefined steroid-sparing) have been validated as surrogates of therapeutic success against mortality, end-stage renal disease, or other organ failures?
Surrogates are defined here with the strict definition of not just being a predictor, but also able to explain a treatment effect in an adjusted analysis. An ideal surrogate is in the pathway that connects an intervention with its clinical outcome effect; the intervention has no effect that is exercised by pathways that are not captured by the surrogate; moreover, the surrogate has no other connection to the clinical outcome effect that is not mediated by intervention. In this context, the literature review found no single intermediate outcome measure that has been strictly validated as a surrogate marker of therapeutic success against mortality, end-stage renal disease (ESRD), or other organ failures.
Nonetheless, changes in measures of renal function have been associated with renal outcome and ESRD in lupus. In the National Institutes of Health (NIH) lupus nephritis trials comparing prednisone with cyclophosphamide or azathioprine,10–12 doubling of serum creatinine correlated with development of renal insufficiency. In the Lupus Nephritis Collaborative Study patients with persistently elevated serum creatinine levels by week 48 were at higher risk for subsequent renal failure.13 Additionally, response of proteinuria to treatment over 1 year is a predictor of long-term outcomes.14 In the Euro-Lupus Nephritis Trial, decrease in initially elevated serum creatinine and/or proteinuria <1 g/day at 6 months were powerful predictors (odds ratios (ORs) 14.9 and 6.3, respectively) for long-term (median follow-up 41 months) renal outcome.15
Evidence from non-randomised trials supports the use of the Systemic Lupus Erythematosus Disease Activity Index (SLEDAI) and SLEDAI 2000,16–24 Systemic Lupus Activity Measure (SLAM),22 25–29 European Consensus Lupus Activity Measure (ECLAM),22 23 30–32 British Isles Lupus Assessment Group (BILAG),33 34 and SLE index score (SIS) 35–38 disease activity indices as predictors of outcome. Some studies have shown that the baseline disease activity score in a study is not a predictor of subsequent damage or other outcomes.21 39–43 Overall, these activity indices are strong predictors of damage and mortality in long-term observational studies but there is little actual evidence that they can be used as surrogates of treatment effect in RCTs at present.
How do disease activity criteria or other measures (serology, global assessments, organ function tests, or other biomarkers) compare with respect to induction or maintenance of remission and outcome (in general and for specific organs such as renal, neuropsychiatric and skin)?
There are little data on the use of disease activity indices in RCTs, as they have been developed and validated in the context of long-term observational studies.44–46 In a controlled trial of intravenous cyclophosphamide (IVC) vs intravenous methylprednisolone (MP) in severe neuropsychiatric SLE, cyclophosphamide treatment resulted in significantly higher response rate, and this was followed by beneficial effects on SLEDAI and a steroid-sparring effect.47 In a RCT of pulse IVC vs mycophenolate mofetil (MMF) in the induction therapy of proliferative lupus nephritis, remission occurred in 13 out of 25 patients (52%) in the IVC group and 11 out of 19 patients (58%) in the MMF group, and this was followed by similar improvement in the SLEDAI between the 2 groups.48 Another RCT comparing ciclosporin and azathioprine for maintenance therapy in diffuse lupus nephritis demonstrated equal efficacy of the two agents in the prevention of disease flares and reductions in the SLAM score.49 By contrast, changes in ECLAM could not differentiate “good” from “poor” long-term renal outcome patients in the Euro-Lupus Nephritis Trial.15
Data from RCTs 48 50 51 and observational cohorts52–55 indicate that serum C3 concentrations correlate with renal outcomes. In a prospective study of 189 Chinese patients with SLE with diffuse proliferative nephritis, persistently low C3 was associated with a hazard ratio (HR) of 3.8 (95% CI 1.0 to 13.9) for nephritic renal flare.54 Similarly, low serum C4 has been associated with renal relapse or adverse renal outcomes51 53–57 (eg, low C4 at the time of response was associated with a likelihood ratio (LR) 14.2 (95% CI 4.7 to 42.5) for future renal flares).56
Reduction of anti-dsDNA titres has been associated with induction of remission in RCTs51 56 58 and prospective cohort studies.52 53 55 59 60 Detectable anti-dsDNA at the time of complete renal response have been associated with a LR 4.9 (95% CI 2.0 to 12.1) for subsequent renal flare.56 However, in one study a decrease in anti-dsDNA antibodies was observed before SLE flares (defined by SLEDAI), including renal flares.61
Data from several RCTs13 15 48 50 51 56 and non-randomised studies31 62 indicate that serum creatinine concentrations correlate strongly with induction/maintenance of renal remission and development of ESRD. Patients participating in nephritis trials with serum creatinine >2.0 mg/dl at the time of response had a LR 10.8 (95% CI 2.7 to 42.3) for development of ESRD.56 Additionally, patients with decreased serum creatinine at 6 months after treatment had an OR 14.9 (95% CI 2.0 to 111.8) for a favourable long-term renal outcome.15 Other measures of renal function such as doubling of serum creatinine10–12 63 and creatinine clearance53 62 64 also correlate with induction of renal remission and outcome.
Proteinuria is a strong determinant for induction of renal remission, renal flares and outcome.12 14 15 31 48 50–52 56 63–66 In patients followed-up for a median 73 months reduction of proteinuria <1 g/day at 6 months has been associated with good long-term renal outcome (OR 6.3, 95% CI 1.2 to 34.4).15
What is the concordance of quality of life measures and of physical function measures with other outcomes in patients with lupus? What is the concordance between patient self-assessment with non-patient-appraised outcome measures?
Quality of life (QoL) and physical function measures inversely correlated to a limited extent with disease activity and damage scores in some observational studies (correlation coefficients varying from −0.20 to −0.40).67–74 Quite a few studies however, have found no significant associations between QoL measures and disease activity or damage.75–84 The Short Form (SF)-36 health survey has been the most studied QoL measure, although other measures have also been used. There is only modest agreement between patient-assessed and doctor-assessed disease activity with correlation coefficients ranging from 0.20 to 0.30.74 77–79 84–87 This discordance may result from patients scoring their disease activity based on their psychological and physical well-being, whereas doctors are based on the clinical and physical signs and symptoms and on laboratory abnormalities of SLE.85 87
Does early (6 months) response in intermediate outcomes concord with late response in these intermediate outcomes?
Data from RCTs and observational cohorts show that early response in intermediate outcome measures (specifically serum creatinine,15 50 88 proteinuria,15 38 50 53 57 88–90 anti-dsDNA titres,38 53 57 88 91 complement C3,15 38 50 53 57 88 90 91 prednisone dose,38 47 90 92 and SLEDAI38 47 92 and ECLAM38 90 scores) correlates with later response in these outcome measures. These data, however, are indirect and are based on the sustained improvement in intermediate outcome measures.
What is the evidence for the minimum clinically meaningful effect that should be demonstrated using activity indices, renal or skin or QoL measures, to claim efficacy in a trial?
There was no direct evidence for the definition of minimum clinically meaningful effect in any of the outcome measures considered, sufficient to claim efficacy in a clinical trial. Expert-derived rules have been developed for clinically meaningful differences in specific instruments, including the SLEDAI, SLAM, BILAG, SIS indices44 and the American College of Rheumatology (ACR) ad hoc committee on renal response criteria for lupus nephritis trials,93 but they have yet to be validated in RCTs. In addition, only few RCTs48 49 58 65 90 94–97 in SLE have stated power calculations and they do not make assumptions about the minimum clinically meaningful effect in intermediate outcome measures.
What measures have been used to record harms in SLE trials?
The systematic literature search found no standardised approach to the recording of harms (adverse events) in SLE trials.
What is the evidence that blinded studies perform similarly as non-blinded studies for the respective intermediate outcome measures?
This is important for interpreting the non-blinded outcome and validation studies performed in SLE. Non-blinded assessments are often used to determine outcome measures for clinical trials in SLE. There have been very few blinded studies conducted to date in SLE, eg, methotrexate,98 leflunomide,99 dehydroepiandrosterone (DHEA),100–105 LJP394,59 106 107 and anti-CD40 ligand,108 and no formal comparison with non-blinded studies exists. Based on evidence from other medical fields, it is possible that unmasked trials may occasionally be biased towards more optimistic results than double-blind trials, but this has not been shown consistently across all fields.109
How do outcomes compare between patients with SLE participating in clinical trials and those in observational cohorts?
There is no direct outcome comparison of patients participating vs non-participating in specific trials or observational cohorts. Disease progression rates and outcome rates in RCTs, nevertheless, are in the same range as those observed in observational cohorts, adjusting for predictors of disease severity, activity and damage; however, indirect comparisons are precarious.
DISCUSSION
There is increasing interest in developing more specific therapies with improved efficacy and less harm over traditional drug treatments. SLE is a complex multi-organ disease with many relapses and remissions during its course, and designing a high-quality RCT poses many challenges. Early trials by the NIH exploring the use of cytotoxic agents in lupus nephritis implemented the well-defined end-point of renal failure or ESRD. Because of the global nature of the disease, the long-term follow-up required to realise these outcomes, and the relatively low frequency with which they occur, there is an unmet need to validate instruments which assess global and specific disease activity and outcome in short-term to intermediate-term clinical trials.110
Our systematic literature search concluded that there is no strict evaluation of any intermediate outcome measure (including disease activity indices) as surrogate of therapeutic success against mortality, ESRD, or other organ failure. Thus, we can only make suggestions on what outcomes may be considered, not absolute recommendations or guidelines that should be strictly followed.
There are several biochemical, serological and immunological abnormalities that characterise active disease; yet, initiation or changes in therapy are not based solely on these parameters. Moreover, the superiority of one intervention against another cannot be claimed based on changes in biomarkers. Therefore, there is a need to accumulate long-term evidence using more hard clinical endpoints for SLE-related interventions.
There is no standardised approach to the recording of harms in SLE trials, and this creates problems in measuring benefit-to-risk ratios from interventions. Standardised harms data collection and reporting in the future would enable comparison of clinical trial data, especially as safety (differences in harms) may become an increasingly important discriminatory point between drugs with equal efficacy.
In summary, this review is a comprehensive summary of the current evidence available for outcome measures included in SLE trials, and formed the basis for the development of the EULAR recommendations for points to consider in clinical trials in SLE.8
Acknowledgments
This work is dedicated to the memory of Dr Jose Font, a member of the committee. We thank Dr Maxime Dougados for his encouragement, guidance and support. We also thank the staff of the EULAR House in Zurich (Fred and Elly Wyss, Robert Buff and Ernst Isler, and their associates) for their warm hospitality and outstanding organisation.
REFERENCES
Footnotes
Competing interests: None declared.
Funding: Support for this work was provided via a grant from the EULAR Executive Committee.