Objective To assess the validity of the 6 min walk test (6MWT) in pulmonary arterial hypertension secondary to systemic sclerosis (PAH-SSc) according to the OMERACT filter.
Methods A systematic literature search was conducted from 1966 through June 2009. The assessment of validation of the 6MWT was based on the OMERACT filter criteria with the domains ‘truth’, ‘discrimination’ and ‘feasibility’.
Results From the 57 articles identified, 9 (16%) were analysed. The 6MWT had face validity as it has been accepted by the registration agencies as a surrogate of PAH-SSc. It was sensitive to change in response to therapy, with effect sizes ranging from 0.30 to 1.37 with a parallel variation of haemodynamic parameters measured by right heart catheterisation. Feasibility was also validated. Content validity was not confirmed as this test was not specific for PAH-SSc. There was some evidence that 6MWT might meet criterion/construct validity, reproducibility and sensitivity to change over time, but insufficient data were provided to fully validate these components.
Conclusions Current evidence suggests partial validation of the 6MWT in PAH-SSc according to the OMERACT filter. Further dedicated studies are needed to validate completely the 6MWT in PAH-SSc, taking into account the comorbidities interfering with the 6MWT.
Statistics from Altmetric.com
Pulmonary arterial hypertension (PAH) has emerged as one of the leading causes of death in systemic sclerosis (SSc).1 Despite therapeutic advances which improved the prognosis of this condition, outcome measures for PAH are poorly defined and those available are only partially validated or considered as not optimal.2 Indeed, the evaluation of PAH is made difficult by other cardiopulmonary comorbidities associated with SSc (ie, interstitial lung disease or primary myocardial involvement). Recently, 78 experts in PAH-SSc from the OMERACT-EPOSS (Expert Panel on Outcome Measures in PAH related to SSc) Group defined a core set of outcome measures using the Delphi consensus methods.3 Among these tools, the 6 min walk test (6MWT) is the most widely used as a primary outcome to assess therapeutic efficacy in PAH clinical trials and is accepted by the Food and Drug Administration. Although this outcome measure is recommended by this expert group, it is necessary to ascertain if the 6MWT is validated in PAH-SSc since its aetiology and prognosis are not the same as idiopathic pulmonary arterial hypertension. The aim of this study was to analyse the status of validation of the 6MWT in SSc according to the criteria of the OMERACT filter.4 Different from other approaches performed several years ago, we used an evidence-based systematic literature analysis for this assessment and included recent primary research articles from 1966 through June 2009, taking into consideration the large number of published articles in the field during recent years.
Full details of the methodology are shown in the online data supplement.
Systematic literature review
Studies in which the 6MWT was used for the evaluation of patients with PAH/PH-SSc were searched in PubMed and the Cochrane Controlled Trial Register, as were relevant references from retrieved articles during the last 2 years. Inclusion and exclusion criteria emphasised articles in which patients with SSc-PAH were predominant or could be analysed separately and required ≥5 patients.
Evidence was assessed with respect to the level of the evidence and the quality of randomised controlled trials (RCTs) was assessed according to the Jadad criteria.5 6 The quality criteria according to the definition of pulmonary hypertension are summarised in table 1.
Application of the OMERACT filter
The OMERACT process involves consensus on outcome measures and is based on the ‘OMERACT filter’ which is composed of three key components: truth (face, content and criterion/construct validity), discrimination (reliability, sensitivity to change and discriminant capacity in response to therapy) and feasibility.4
Results of the systematic literature search
Quality assessment of retrieved articles
Five studies were RCTs (level 1b),7 10 13,–,15 one was a longitudinal study with comparison to a historical control group (level 2b)12 and three were descriptive studies (level 3).8 9 11 The mean±SD Jadad score for RCTs was 4.1±1.0. Seven studies included well-defined PAH-SSc subgroups according to quality level A17 9 10 12,–,15 and two studies were classified C18 11 (see table in online supplement).
Status of validation according to the OMERACT criteria for the 6MWT
An agreement by a group of experts is required to define the appropriateness of 6MWT to be used for the evaluation of PAH (table 2). As it was accepted by the registration agencies of the USA and Europe as a surrogate of PAH and selected by the EPOSS Group as part of the core set of outcome measures for SSc, this test clearly has face validity.
6MWT did not pass this specific filter as no paper evaluated 6MWT in a wide-ranging and comprehensive group of patients with SSc. Moreover, indirect evidence suggests that the 6MWT is not specific for PAH-SSc-related exercise impairment. Thus, in three different studies, 6MWT was also associated in multivariate analysis with measures of myocardial involvement, interstitial lung disease, musculoskeletal disorders and inflammation.8 9 11 While there is no study available in SSc-PAH with the highest quality level A1, the expert group agreed that, based on the evidence from non-PAH studies, the 6MWT will very likely not meet this criterion and rated content validity as not validated.
Criterion validity/construct validity
Since the 6MWT was never tested against the external or ‘gold’ standard (right heart catheterisation (RHC)) in any study of patients with SSc-PAH, the 6MWT does not meet this aspect of validation. As an alternative, correlations with other validated parameters were searched to estimate concurrent and convergent validity as surrogates for criterion validity and as indicators of overall construct validity. One study found that 6MWT was significantly associated with mortality in univariate analysis. Survival increased by 24% for every 100 m further a patient could walk (HR 0.76, 95% CI 0.58 to 0.98, p=0.037). However, this result was not confirmed in the multivariate analysis. In this study the 6MWT was not an independent risk factor for decreased survival in patients with PAH-SSc (HR 0.81, 95% CI 0.59 to 1.12, p=0.20).12 At this time the 6MWT has therefore not met the test of criterion/construct validity, but it was rated partially validated for construct validity by the expert group as there is some evidence that it might correlate with survival.
Insufficient data were provided in the analysed studies to assess the reliability and reproducibility of the 6MWT in the specific situation of PAH-SSc (κ coefficient and intraclass and interclass correlation coefficients were not assessed). However, because tests in SSc-ILD indicated reproducibility, it was rated partially validated.16 It was not possible to assess the sensitivity to change over time because there were no suitable data to analyse. As an alternative, we assessed the sensitivity to change of the 6MWT in the placebo groups of five high-quality RCTs.7 10 13,–,15 These studies provided data on exercise capacity for 100 patients included in the placebo groups. Unfortunately, there were insufficient data to calculate the standardised response mean in all studies and thus complete a meta-analysis. However, it is noteworthy that, over time, there was a deterioration in the mean distance walked in this specific group in four studies (deterioration from 2.6 m to 38 m) and no variation in one study (improvement of 3±8 m). Thus, with regard to these data, the 6MWT was rated by the experts as at least partially validated.
Discriminant capacity in response to therapy was validated in the five high-quality RCTs described above. These studies provided data on exercise capacity for 283 patients with SSc. The mean trial duration was 13±4.0 weeks. The 6MWT was sensitive to change in response to therapy with effect sizes ranging from 0.30 to 1.37 (table 3). Moreover, there was also a parallel variation between the effect sizes of the 6MWT and the effect sizes of haemodynamic parameters (mean pulmonary artery pressure (mPAP) and pulmonary vascular resistance (PVR)) during treatment, as measured by RHC in the analysed trials (table 3).
The 6MWT is easy to perform and values are easy to interpret with no special equipment required; minimal training is required and patient/physician acceptance is good. It is also an inexpensive test that requires a minimum number of healthcare staff and can be performed in an office setting.11 Thus, availability of the 6MWT at investigational sites and its practicality both in specialised and community practices make it an eminently feasible test.17 As a limitation, there might be rare disease-related manifestations such as arthritis or severe skin fibrosis where the test cannot be performed.
This is the first study to address the validity of 6MWT as an outcome measure in PAH-SSc, with respect to the OMERACT filter, using a systematic literature review. Current evidence suggests that the 6MWT passes the filter for face validity, discriminant capacity with treatment and feasibility. However, content validity is not validated and no robust data allow full confirmation of construct/criterion validity, reliability and sensitivity to change over time (table 3).
The heterogeneity of the trials limited the analysis to only a few studies with patients with PAH-SSc diagnosed according to the highest quality level A1. Many studies included patients with pulmonary hypertension related to PAH, left heart disease and severe interstitial lung disease, without subgroup analysis which allowed one to examine SSc-PAH per se.
Content validity was considered by the expert group as not valid, although one study showed that impairment of exercise performance was associated with pulmonary hypertension.18 However, this study had a low C2 quality level and was excluded from the initial analysis as exercise capacity was not assessed by the 6MWT. Moreover, recent data have shown that patients with SSc judged free of both cardiac and pulmonary diseases had reduced exercise capacity, suggesting that some SSc-specific or non-specific comorbidities independent of cardiopulmonary involvement, such as articular/soft tissue involvements, may influence exercise capacity in SSc.8 19 20 This has an impact on future trial design in patients with SSc-PAH. If the 6MWT is chosen as an outcome, patients with significant comorbidities either need to be excluded or need to be controlled for by appropriate tests.
Despite insufficient data to assess the reliability and reproducibility of the 6MWT in PAH-SSc per se, follow-up analyses by Buch et al16 showed the high reproducibility of the 6MWT in patients with SSc-ILD. The within-subject intertest Pearson correlation coefficient on repeated 6MWT was reported to be 0.95 (p<0.001). Based on all of the above, reproducibility was rated by experts as partially validated.
We used placebo groups of RCTs to assess the sensitivity to change over time. One could argue that placebo is a form of intervention and may be a bias. This explains why experts rated this item as partially validated (table 2). However, it appears likely that the 6MWT also shows sensitivity to change without placebo intervention. Although the 6MWT had discriminant capacity with treatment, this result has to be taken cautiously because most of the effect sizes calculated were not significant. This may have been related to the variability of the testing procedures or to the low number of patients with SSc included (only five trials allowed subgroup analyses of pivotal RCTs on approximately 20–25% of the total population). On the other hand, two high-quality RCTs, performed in a population of patients with PAH-SSc with sufficient power to detect a significant difference, clearly showed the sensitivity to change of the 6MWT when the patients were treated with epoprostenol and treprostinil.13
This structured literature review and the assessment of identified papers according to the OMERACT filter revealed several aspects of 6MWT that need further validation in additional studies (criterion/construct validity, reliability and sensitivity to change over time). Although the OMERACT filter is not an obligatory framework to assess the full validation of 6MWT as an outcome measure in PAH-SSc, it defines a specific research agenda which needs to be addressed.
Definite validation of construct/criterion validity of the 6MWT requires significant correlation between the gold standard RHC and the 6MWT specifically in PAH-SSc cross-sectional studies.
Validation of reliability of the 6MWT in PAH-SSc requires comparisons of repeated 6MWT assessments performed within a short time period by the same investigator (intraobserver variability) and by two independent investigators (interobserver variability) at the same time in patients with well-defined PAH-SSc.
The type of studies required to assess the validity of 6MWT with respect to the sensitivity to change are longitudinal studies including patients with PAH-SSc without treatment and parallel 6MWT and RHC evaluations at different time points.
Our study did not assess the 6MWT in patients with heart or lung disease in general. Further dedicated studies are thus needed to assess the 6MWT in patients with SSc with interstitial lung disease or primary myocardial involvement.
In conclusion, this systematic literature analysis showed that 6MWT in PAH-SSc fulfills the OMERACT criteria only for face validity, discriminant capacity with treatment and feasibility. Content validity was considered as not valid. Further studies are therefore needed to completely validate the 6MWT in PAH-SSc for construct/criterion validity, reliability and sensitivity to change.
JA, OK-B, OD and DEF contributed equally to this work.
Provenance and peer review Not commissioned; externally peer reviewed.
Competing interests None.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.