Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Clinicians are encouraged to practice “evidence-based medicine”, a term implying that a practitioner can access, summarise, and apply information from the literature to day to day clinical problems and questions.1 But how convincing should the evidence be for a clinician to accept the data and adopt the approach into his or her practice? Undoubtedly the double blind, placebo controlled, randomised trial remains as the ultimate tool for gathering the strongest evidence of therapeutic efficacy and safety. However, this study design is at times unfeasible, unethical or may not even be appropriate for obtaining the information needed (for example, long term outcome), and less stringent alternatives have to be considered.2 ,3 In rheumatology, where the optimal therapeutic approach to many conditions is still uncertain, approved drugs are frequently used “off label” (implying that the agent is being used to treat a disease for which it has not gained formal approval, or at a dosage level other than that recommended on the label). Diverse combinations of drugs are also commonly tried as clinicians attempt to retain or reclaim a higher functional outcome and quality of life for their patients. Understandably, one frequently encounters conflicting data when searching for evidence about the clinical utility of these new approaches.4 Furthermore, the likelihood of a pivotal trial ever being done to establish unequivocal evidence of effectiveness and safety may be remote.
Less rigorous designs may not produce evidence as compelling to the practitioner as a pivotal study, but they can provide at least some insight into therapeutic effectiveness and safety where otherwise none would exist. One such approach is the open, randomised, actively controlled trial (ORACT), a mainstay of haematology-oncology research studies of the US based Pediatric Oncology Group (POG) and Children’s Cancer Study Group.5 Virtually all newly diagnosed children with cancer are randomly assigned to one of two or more active treatment arms of approved drugs and continue “on-protocol” for the duration of their follow up. Thus, each child becomes a resource of information for future patients (particularly crucial in rare conditions) as their data are entered into the constantly expanding database.
The premise of this leader is that the ORACT design offers an opportunity to conduct studies in rheumatology that would otherwise not be possible.
What can ORACTs offer the field of rheumatology?
This design offers several distinct advantages over more stringent designs in terms of cost, “user friendliness”, regulatory requirements, enrollment periods, logistics, and statistical approaches to the data.
ORACTs can be done without funding from a sponsor because the costs of the trial are borne largely by public or private health insurance or by the patient directly. This is possible because the trial uses previously approved drugs, neither the investigator nor the patient is blinded to the treatment, and follow up procedures are identical to those necessitated under routine medical care. Some cost is incurred because a central coordinating centre is necessary to manage the trial and, in some countries, a small honorarium may be paid to the health professionals involved in the study. These funds must come from an external source such as professional or charitable organisations or public funds.
ORACTs can be made more “user friendly” than blinded placebo controlled trials. Obtaining adequate sample size is a problem in practically all rheumatic disease trials. ORACTs can be designed such that the eligibility criteria, concurrent medication restrictions, and dosing schedules are more “relaxed”, thus permitting enrollment of a broader spectrum of patients and retention of a higher percentage of enrolled subjects in the trial. As a protocol becomes less stringent, patients and physicians typically become more accepting of it and participate to a greater degree.
Regulatory issues that often delay the start of a trial usually can be avoided. Because the drugs being used are already approved, any physician may prescribe them, even if for an “off-label” use. Thus, application to regulatory agencies to conduct the trial is not usually necessary. The trial must, of course, still obtain the approval of the appropriate human subjects committees.
The enrollment period can be long, thus permitting sufficient numbers of patients to be accumulated for studies of rare diseases such as linear scleroderma and dermatomyositis. Clinicians participating in trials sponsored by conventional means often are strongly encouraged to enter as many patients as quickly as possible so that submission deadlines are met or before grant funding expires. As ORACTs are not designed to seek regulatory agency approval, and no sponsor is rushing for completion, the enrollment period can be extended.
Logistical and ethical difficulties in keeping a study blinded are avoided. If the study is multi-national, the shipping of blinded medication across international boundaries and conforming to local regulations about how blinded medication is to be handled produce a logistical and financial nightmare for the managers of the trial. Maintaining the double blind until the last patient completes the study can be nearly impossible, especially in studies involving children. Anxious parents want to know what the child received during the trial so that it can be continued if it worked or something else tried if it didn’t. Methods do exist for accommodating this wish, but this does not necessarily help the paediatrician confronted with the mother who demands to know the day her child completes the trial exactly which drug was given.
Analysts are free to use non-standard statistical approaches to the data. For example, Bayesian analysis provides a means for the incorporation of prior beliefs based on previous trial data and clinical experience. p Values and confidence intervals—mainstays of standard statistical approaches (but largely dependent upon sample size and “more extreme events that didn’t actually occur”6)—can be avoided in favour of a simple Bayesian answer to the question, “What is the probability that a treatment is superior based on the observed data and prior beliefs?”
When should ORACTs be considered?
ORACTs should never be the “default” study design, but instead should be considered only when more rigorous designs are deemed impractical or unethical. In addition, before considering the ORACT design, there should be at least some anecdotal evidence that the approach/es may be effective without unacceptable toxicity. Several situations in which the ORACT design might be appropriate are as follows.
Funding opportunities are remote for a trial in which there is substantial anecdotal evidence that the treatment has a degree of effectiveness. Funding difficulties are routinely encountered when combination therapy trials are considered because they will not lead to a new indication for a potential pharmaceutical sponsor’s previously approved drug. An example is the question of the comparative safety and effectiveness of the repetitive use of pulse therapy with methylprednisolone and cyclophosphamide and oral methotrexate versus pulse methylprednisolone and oral methotrexate alone in patients with active systemic juvenile chronic arthritis.7
When the disease to be studied is rare and a prolonged enrollment period is required to gather sufficient numbers of patients. The protracted enrollment as well as limited market are powerful disincentives for potential sponsors.
It is not possible or ethical to (a) give placebo, or (b) blind the investigators and the patient as to which treatment is being received. An example is sham infusion trials, which appear to be close to extinction because of the risk of infection and invasiveness of a worthless procedure.
Meta-analysis is not possible because of a lack of suitable studies to distill into a “best” estimate about the specific clinical question. This technique has been found useful even when the studies included in the meta-analysis are small and there exists considerable variation in methods (although the severe limitations of meta-analyses are now becoming increasingly apparent8 ,9).
What to look for in an ORACT report
Rigorous scientific methodology can be designed into these trials, and clinicians will find it useful to consult any of a number of excellent guides for determining the overall quality and validity of a trial.10 ,11 For example, answers to the following questions should be evident.
Were all patients who entered the trial properly accounted for and attributed at its conclusion?
Were the groups similar at baseline?
Apart from the experimental intervention, were the groups treated equally?
Because these trials are open and many are multi-centred (and, increasingly, multi-national), the following points should also be evident.
Did a central coordinating centre, staffed with persons who are experienced in the design, conduct, and reporting of clinical trial data, manage the trial? Chief functions of the coordinating centre include protocol and data collection form preparation, facilitation of human subjects approval, assignment of randomisation numbers, monitoring of the participating clinical centres, electronic data collection, management, analysis and reporting.
Did all centres use a standardised protocol and data collection forms that were agreed to by each investigator? This includes a standardised method for recording the nature, severity, and attributability of adverse events.
Did the investigator or the patient have any influence or knowledge as to which of the treatment arms of the protocol that the patient was to receive prior to randomisation?
Were the response variables that served as the primary measures of effectiveness (such as the core set for juvenile arthritis trials12) as objective as possible and did all investigators assess the outcome using the same technique?
If questionnaires were used to measure functional ability or quality of life, were they cross culturally adapted, translated, and validated within each culture or country in which they were utilised?
Was there an external monitoring committee (similar to that used by the POG and CCSG13) that assessed the data routinely in order to make decisions about trial continuation?
What are the shortcomings of the randomised open trial approach?
Quite obviously these trials have pitfalls, the largest of which include the following.
Non-blinded assessment of response and adverse effects. Not masking either the physician or patient to the treatment always has the potential for the introduction of substantial bias. It has been suggested that the use of an independent blinded assessor, (one who evaluates the patient’s response without knowledge of the treatment being given) can reduce the amount of bias. However, in practice this approach is often untenable because many rheumatology centres, particularly paediatric, are one physician operations. The blinded assessor ideally should be the same appropriately qualified person for the duration of follow up, and this presents additional logistical problems.
Laboratory tests that are not considered part of routine care may have to be avoided in some countries because the costs of patient procedures and services are to be covered by insurance or self pay. However, externally funded research programmes may provide a way for carrying out adjunctive laboratory science investigations.
None of the treatment arms may be any better than placebo even though one or more show a strongly positive response. The ethics of the use of a placebo control group in any disease in which some standard of care exists is being actively debated and early resolution of the issue seems remote.14 Nevertheless, placebo response in rheumatology clinical trials is notoriously high, especially in paediatric trials. If none of the treatments has been compared with placebo, and a placebo arm is not included, then one may conclude erroneously that the treatments are equally effective when, in fact, they are equally non-effective. It follows then that, whenever possible, treatments known to be effective for a condition should be used as one of the treatment arms.
“Off-label” use of the drug is encouraged if the results are positive.
How did open, actively controlled trials become well accepted in haematology-oncology and not in rheumatology? The chief reason is probably the objectiveness of the outcome variables: the validity and reliability of the assessment of pain in the 7 year old child with juvenile arthritis is quite different from determining the two year year survival rate among those with neuroblastoma. In addition to the objectiveness of the outcomes, perhaps the difference in the morbidity and mortality of cancer has made oncologists less willing than rheumatologists to use placebo controls. Whatever the reason, as even actively controlled trials become more difficult to do because of ethical and financial concerns,15 and evidence of the potential benefits of combination therapy accumulates,4alternatives to the placebo controlled double blind design must be found. Properly designed ORACTs may provide one such alternative.