Randomized controlled trial explained

A randomized controlled trial (or randomized control trial;^[1] RCT) is a form of scientific experiment used to control factors not under direct experimental control. Examples of RCTs are clinical trials that compare the effects of drugs, surgical techniques, medical devices, diagnostic procedures, diets or other medical treatments.^{[2] [3]}

Participants who enroll in RCTs differ from one another in known and unknown ways that can influence study outcomes, and yet cannot be directly controlled. By randomly allocating participants among compared treatments, an RCT enables statistical control over these influences. Provided it is designed well, conducted properly, and enrolls enough participants, an RCT may achieve sufficient control over these confounding factors to deliver a useful comparison of the treatments studied.

Definition and examples

An RCT in clinical research typically compares a proposed new treatment against an existing standard of care; these are then termed the 'experimental' and 'control' treatments, respectively. When no such generally accepted treatment is available, a placebo may be used in the control group so that participants are blinded to their treatment allocations. This blinding principle is ideally also extended as much as possible to other parties including researchers, technicians, data analysts, and evaluators. Effective blinding experimentally isolates the physiological effects of treatments from various psychological sources of bias.

The randomness in the assignment of participants to treatments reduces selection bias and allocation bias, balancing both known and unknown prognostic factors, in the assignment of treatments.^[4] Blinding reduces other forms of experimenter and subject biases.

A well-blinded RCT is considered the gold standard for clinical trials. Blinded RCTs are commonly used to test the efficacy of medical interventions and may additionally provide information about adverse effects, such as drug reactions. A randomized controlled trial can provide compelling evidence that the study treatment causes an effect on human health.^[5]

The terms "RCT" and "randomized trial" are sometimes used synonymously, but the latter term omits mention of controls and can therefore describe studies that compare multiple treatment groups with each other in the absence of a control group.^[6] Similarly, the initialism is sometimes expanded as "randomized clinical trial" or "randomized comparative trial", leading to ambiguity in the scientific literature.^{[7] [8]} Not all RCTs are randomized controlled trials (and some of them could never be, as in cases where controls would be impractical or unethical to use). The term randomized controlled clinical trial is an alternative term used in clinical research;^[9] however, RCTs are also employed in other research areas, including many of the social sciences.

History

The first reported clinical trial was conducted by James Lind in 1747 to identify a treatment for scurvy.^[10] The first blind experiment was conducted by the French Royal Commission on Animal Magnetism in 1784 to investigate the claims of mesmerism. An early essay advocating the blinding of researchers came from Claude Bernard in the latter half of the 19th century. Bernard recommended that the observer of an experiment should not have knowledge of the hypothesis being tested. This suggestion contrasted starkly with the prevalent Enlightenment-era attitude that scientific observation can only be objectively valid when undertaken by a well-educated, informed scientist.^[11] The first study recorded to have a blinded researcher was published in 1907 by W. H. R. Rivers and H. N. Webber to investigate the effects of caffeine.^[12]

Randomized experiments first appeared in psychology, where they were introduced by Charles Sanders Peirce and Joseph Jastrow in the 1880s,^[13] and in education.^{[14] [15] [16]} The earliest experiments comparing treatment and control groups were published by Robert Woodworth and Edward Thorndike in 1901,^[17] and by John E. Coover and Frank Angell in 1907.^{[18] [19]}

In the early 20th century, randomized experiments appeared in agriculture, due to Jerzy Neyman^[20] and Ronald A. Fisher. Fisher's experimental research and his writings popularized randomized experiments.^[21]

The first published Randomized Controlled Trial in medicine appeared in the 1948 paper entitled "Streptomycin treatment of pulmonary tuberculosis", which described a Medical Research Council investigation.^{[22] [23] [24]} One of the authors of that paper was Austin Bradford Hill, who is credited as having conceived the modern RCT.^[25]

Trial design was further influenced by the large-scale ISIS trials on heart attack treatments that were conducted in the 1980s.^[26]

By the late 20th century, RCTs were recognized as the standard method for "rational therapeutics" in medicine.^[27] As of 2004, more than 150,000 RCTs were in the Cochrane Library.^[25] To improve the reporting of RCTs in the medical literature, an international group of scientists and editors published Consolidated Standards of Reporting Trials (CONSORT) Statements in 1996, 2001 and 2010, and these have become widely accepted.^[4] Randomization is the process of assigning trial subjects to treatment or control groups using an element of chance to determine the assignments in order to reduce the bias.

Ethics

Although the principle of clinical equipoise ("genuine uncertainty within the expert medical community... about the preferred treatment") common to clinical trials^[28] has been applied to RCTs, the ethics of RCTs have special considerations. For one, it has been argued that equipoise itself is insufficient to justify RCTs.^[29] For another, "collective equipoise" can conflict with a lack of personal equipoise (e.g., a personal belief that an intervention is effective).^[30] Finally, Zelen's design, which has been used for some RCTs, randomizes subjects before they provide informed consent, which may be ethical for RCTs of screening and selected therapies, but is likely unethical "for most therapeutic trials."^{[31] [32]}

Although subjects almost always provide informed consent for their participation in an RCT, studies since 1982 have documented that RCT subjects may believe that they are certain to receive treatment that is best for them personally; that is, they do not understand the difference between research and treatment.^{[33] [34]} Further research is necessary to determine the prevalence of and ways to address this "therapeutic misconception".^[34]

The RCT method variations may also create cultural effects that have not been well understood.^[35] For example, patients with terminal illness may join trials in the hope of being cured, even when treatments are unlikely to be successful.

Trial registration

In 2004, the International Committee of Medical Journal Editors (ICMJE) announced that all trials starting enrolment after July 1, 2005, must be registered prior to consideration for publication in one of the 12 member journals of the committee.^[36] However, trial registration may still occur late or not at all.^{[37] [38]} Medical journals have been slow in adapting policies requiring mandatory clinical trial registration as a prerequisite for publication.^[39]

Classifications

By study design

One way to classify RCTs is by study design. From most to least common in the healthcare literature, the major categories of RCT study designs are:^[40]

• Parallel-group – each participant is randomly assigned to a group, and all the participants in the group receive (or do not receive) an intervention.^{[41] [42]}
• Crossover – over time, each participant receives (or does not receive) an intervention in a random sequence.^{[43] [44]}
• Cluster – pre-existing groups of participants (e.g., villages, schools) are randomly selected to receive (or not receive) an intervention.^{[45] [46]}
• Factorial – each participant is randomly assigned to a group that receives a particular combination of interventions or non-interventions (e.g., group 1 receives vitamin X and vitamin Y, group 2 receives vitamin X and placebo Y, group 3 receives placebo X and vitamin Y, and group 4 receives placebo X and placebo Y).

An analysis of the 616 RCTs indexed in PubMed during December 2006 found that 78% were parallel-group trials, 16% were crossover, 2% were split-body, 2% were cluster, and 2% were factorial.^[40]

By outcome of interest (efficacy vs. effectiveness)

See main article: Pragmatic clinical trial. RCTs can be classified as "explanatory" or "pragmatic."^[47] Explanatory RCTs test efficacy in a research setting with highly selected participants and under highly controlled conditions.^[47] In contrast, pragmatic RCTs (pRCTs) test effectiveness in everyday practice with relatively unselected participants and under flexible conditions; in this way, pragmatic RCTs can "inform decisions about practice."^[47]

By hypothesis (superiority vs. noninferiority vs. equivalence)

Another classification of RCTs categorizes them as "superiority trials", "noninferiority trials", and "equivalence trials", which differ in methodology and reporting.^[48] Most RCTs are superiority trials, in which one intervention is hypothesized to be superior to another in a statistically significant way.^[48] Some RCTs are noninferiority trials "to determine whether a new treatment is no worse than a reference treatment."^[48] Other RCTs are equivalence trials in which the hypothesis is that two interventions are indistinguishable from each other.^[48]

Randomization

The advantages of proper randomization in RCTs include:^[49]

• "It eliminates bias in treatment assignment," specifically selection bias and confounding.
• "It facilitates blinding (masking) of the identity of treatments from investigators, participants, and assessors."
• "It permits the use of probability theory to express the likelihood that any difference in outcome between treatment groups merely indicates chance."

There are two processes involved in randomizing patients to different interventions. First is choosing a randomization procedure to generate an unpredictable sequence of allocations; this may be a simple random assignment of patients to any of the groups at equal probabilities, may be "restricted", or may be "adaptive." A second and more practical issue is allocation concealment, which refers to the stringent precautions taken to ensure that the group assignment of patients are not revealed prior to definitively allocating them to their respective groups. Non-random "systematic" methods of group assignment, such as alternating subjects between one group and the other, can cause "limitless contamination possibilities" and can cause a breach of allocation concealment.

However empirical evidence that adequate randomization changes outcomes relative to inadequate randomization has been difficult to detect.^[50]

Procedures

The treatment allocation is the desired proportion of patients in each treatment arm.

An ideal randomization procedure would achieve the following goals:^[51]

• Maximize statistical power, especially in subgroup analyses. Generally, equal group sizes maximize statistical power, however, unequal groups sizes may be more powerful for some analyses (e.g., multiple comparisons of placebo versus several doses using Dunnett's procedure^[52]), and are sometimes desired for non-analytic reasons (e.g., patients may be more motivated to enroll if there is a higher chance of getting the test treatment, or regulatory agencies may require a minimum number of patients exposed to treatment).^[53]
• Minimize selection bias. This may occur if investigators can consciously or unconsciously preferentially enroll patients between treatment arms. A good randomization procedure will be unpredictable so that investigators cannot guess the next subject's group assignment based on prior treatment assignments. The risk of selection bias is highest when previous treatment assignments are known (as in unblinded studies) or can be guessed (perhaps if a drug has distinctive side effects).
• Minimize allocation bias (or confoundin). This may occur when covariates that affect the outcome are not equally distributed between treatment groups, and the treatment effect is confounded with the effect of the covariates (i.e., an "accidental bias"^{[49] [54]}). If the randomization procedure causes an imbalance in covariates related to the outcome across groups, estimates of effect may be biased if not adjusted for the covariates (which may be unmeasured and therefore impossible to adjust for).

However, no single randomization procedure meets those goals in every circumstance, so researchers must select a procedure for a given study based on its advantages and disadvantages.

Simple

This is a commonly used and intuitive procedure, similar to "repeated fair coin-tossing."^[49] Also known as "complete" or "unrestricted" randomization, it is robust against both selection and accidental biases. However, its main drawback is the possibility of imbalanced group sizes in small RCTs. It is therefore recommended only for RCTs with over 200 subjects.^[55]

Restricted

To balance group sizes in smaller RCTs, some form of "restricted" randomization is recommended.^[55] The major types of restricted randomization used in RCTs are:

• Permuted-block randomization or blocked randomization: a "block size" and "allocation ratio" (number of subjects in one group versus the other group) are specified, and subjects are allocated randomly within each block. For example, a block size of 6 and an allocation ratio of 2:1 would lead to random assignment of 4 subjects to one group and 2 to the other. This type of randomization can be combined with "stratified randomization", for example by center in a multicenter trial, to "ensure good balance of participant characteristics in each group."^[4] A special case of permuted-block randomization is random allocation, in which the entire sample is treated as one block. The major disadvantage of permuted-block randomization is that even if the block sizes are large and randomly varied, the procedure can lead to selection bias.^[51] Another disadvantage is that "proper" analysis of data from permuted-block-randomized RCTs requires stratification by blocks.^[55]
• Adaptive biased-coin randomization methods (of which urn randomization is the most widely known type): In these relatively uncommon methods, the probability of being assigned to a group decreases if the group is overrepresented and increases if the group is underrepresented. The methods are thought to be less affected by selection bias than permuted-block randomization.^[55]

Adaptive

At least two types of "adaptive" randomization procedures have been used in RCTs, but much less frequently than simple or restricted randomization:

• Covariate-adaptive randomization, of which one type is minimization: The probability of being assigned to a group varies in order to minimize "covariate imbalance."^[55] Minimization is reported to have "supporters and detractors" because only the first subject's group assignment is truly chosen at random, the method does not necessarily eliminate bias on unknown factors.^[4]
• Response-adaptive randomization, also known as outcome-adaptive randomization: The probability of being assigned to a group increases if the responses of the prior patients in the group were favorable.^[55] Although arguments have been made that this approach is more ethical than other types of randomization when the probability that a treatment is effective or ineffective increases during the course of an RCT, ethicists have not yet studied the approach in detail.^[56]

Allocation concealment

See main article: Allocation concealment. "Allocation concealment" (defined as "the procedure for protecting the randomization process so that the treatment to be allocated is not known before the patient is entered into the study") is important in RCTs.^[57] In practice, clinical investigators in RCTs often find it difficult to maintain impartiality. Stories abound of investigators holding up sealed envelopes to lights or ransacking offices to determine group assignments in order to dictate the assignment of their next patient.^[58] Such practices introduce selection bias and confounders (both of which should be minimized by randomization), possibly distorting the results of the study.^[58] Adequate allocation concealment should defeat patients and investigators from discovering treatment allocation once a study is underway and after the study has concluded. Treatment related side-effects or adverse events may be specific enough to reveal allocation to investigators or patients thereby introducing bias or influencing any subjective parameters collected by investigators or requested from subjects.

Some standard methods of ensuring allocation concealment include sequentially numbered, opaque, sealed envelopes (SNOSE); sequentially numbered containers; pharmacy controlled randomization; and central randomization.^[58] It is recommended that allocation concealment methods be included in an RCT's protocol, and that the allocation concealment methods should be reported in detail in a publication of an RCT's results; however, a 2005 study determined that most RCTs have unclear allocation concealment in their protocols, in their publications, or both.^[59] On the other hand, a 2008 study of 146 meta-analyses concluded that the results of RCTs with inadequate or unclear allocation concealment tended to be biased toward beneficial effects only if the RCTs' outcomes were subjective as opposed to objective.^[60]

Sample size

See main article: Sample size determination.

The number of treatment units (subjects or groups of subjects) assigned to control and treatment groups, affects an RCT's reliability. If the effect of the treatment is small, the number of treatment units in either group may be insufficient for rejecting the null hypothesis in the respective statistical test. The failure to reject the null hypothesis would imply that the treatment shows no statistically significant effect on the treated in a given test. But as the sample size increases, the same RCT may be able to demonstrate a significant effect of the treatment, even if this effect is small.^[61]

Blinding

See main article: Blinded experiment. An RCT may be blinded, (also called "masked") by "procedures that prevent study participants, caregivers, or outcome assessors from knowing which intervention was received."^[60] Unlike allocation concealment, blinding is sometimes inappropriate or impossible to perform in an RCT; for example, if an RCT involves a treatment in which active participation of the patient is necessary (e.g., physical therapy), participants cannot be blinded to the intervention.

Traditionally, blinded RCTs have been classified as "single-blind", "double-blind", or "triple-blind"; however, in 2001 and 2006 two studies showed that these terms have different meanings for different people.^{[62] [63]} The 2010 CONSORT Statement specifies that authors and editors should not use the terms "single-blind", "double-blind", and "triple-blind"; instead, reports of blinded RCT should discuss "If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how."^[4]

RCTs without blinding are referred to as "unblinded",^[64] "open",^[65] or (if the intervention is a medication) "open-label".^[66] In 2008 a study concluded that the results of unblinded RCTs tended to be biased toward beneficial effects only if the RCTs' outcomes were subjective as opposed to objective;^[60] for example, in an RCT of treatments for multiple sclerosis, unblinded neurologists (but not the blinded neurologists) felt that the treatments were beneficial.^[67] In pragmatic RCTs, although the participants and providers are often unblinded, it is "still desirable and often possible to blind the assessor or obtain an objective source of data for evaluation of outcomes."^[47]

Analysis of data

The types of statistical methods used in RCTs depend on the characteristics of the data and include:

• For dichotomous (binary) outcome data, logistic regression (e.g., to predict sustained virological response after receipt of peginterferon alfa-2a for hepatitis C^[68]) and other methods can be used.
• For continuous outcome data, analysis of covariance (e.g., for changes in blood lipid levels after receipt of atorvastatin after acute coronary syndrome^[69]) tests the effects of predictor variables.
• For time-to-event outcome data that may be censored, survival analysis (e.g., Kaplan–Meier estimators and Cox proportional hazards models for time to coronary heart disease after receipt of hormone replacement therapy in menopause^[70]) is appropriate.

Regardless of the statistical methods used, important considerations in the analysis of RCT data include:

• Whether an RCT should be stopped early due to interim results. For example, RCTs may be stopped early if an intervention produces "larger than expected benefit or harm", or if "investigators find evidence of no important difference between experimental and control interventions."^[4]
• The extent to which the groups can be analyzed exactly as they existed upon randomization (i.e., whether a so-called "intention-to-treat analysis" is used). A "pure" intention-to-treat analysis is "possible only when complete outcome data are available" for all randomized subjects;^[71] when some outcome data are missing, options include analyzing only cases with known outcomes and using imputed data.^[4] Nevertheless, the more that analyses can include all participants in the groups to which they were randomized, the less bias that an RCT will be subject to.^[4]
• Whether subgroup analysis should be performed. These are "often discouraged" because multiple comparisons may produce false positive findings that cannot be confirmed by other studies.^[4]

Reporting of results

The CONSORT 2010 Statement is "an evidence-based, minimum set of recommendations for reporting RCTs."^[72] The CONSORT 2010 checklist contains 25 items (many with sub-items) focusing on "individually randomised, two group, parallel trials" which are the most common type of RCT.

For other RCT study designs, "CONSORT extensions" have been published, some examples are:

• Consort 2010 Statement: Extension to Cluster Randomised Trials^[73]
• Consort 2010 Statement: Non-Pharmacologic Treatment Interventions^{[74] [75]}
• "Reporting of surrogate endpoints in randomised controlled trial reports (CONSORT-Surrogate): extension checklist with explanation and elaboration"^[76]

Relative importance and observational studies

Two studies published in The New England Journal of Medicine in 2000 found that observational studies and RCTs overall produced similar results.^{[77] [78]} The authors of the 2000 findings questioned the belief that "observational studies should not be used for defining evidence-based medical care" and that RCTs' results are "evidence of the highest grade."^{[77] [78]} However, a 2001 study published in Journal of the American Medical Association concluded that "discrepancies beyond chance do occur and differences in estimated magnitude of treatment effect are very common" between observational studies and RCTs.^[79] According to a 2014 (updated in 2024) Cochrane review, there is little evidence for significant effect differences between observational studies and randomized controlled trials.^[80] To evaluate differences it is necessary to consider things other than design, such as heterogeneity, population, intervention or comparator.

Two other lines of reasoning question RCTs' contribution to scientific knowledge beyond other types of studies:

• If study designs are ranked by their potential for new discoveries, then anecdotal evidence would be at the top of the list, followed by observational studies, followed by RCTs.^[81]
• RCTs may be unnecessary for treatments that have dramatic and rapid effects relative to the expected stable or progressively worse natural course of the condition treated.^[82] One example is combination chemotherapy including cisplatin for metastatic testicular cancer, which increased the cure rate from 5% to 60% in a 1977 non-randomized study.^{[82] [83]}

Interpretation of statistical results

Like all statistical methods, RCTs are subject to both type I ("false positive") and type II ("false negative") statistical errors. Regarding Type I errors, a typical RCT will use 0.05 (i.e., 1 in 20) as the probability that the RCT will falsely find two equally effective treatments significantly different.^[84] Regarding Type II errors, despite the publication of a 1978 paper noting that the sample sizes of many "negative" RCTs were too small to make definitive conclusions about the negative results,^[85] by 2005-2006 a sizeable proportion of RCTs still had inaccurate or incompletely reported sample size calculations.^[86]

Peer review

Peer review of results is an important part of the scientific method. Reviewers examine the study results for potential problems with design that could lead to unreliable results (for example by creating a systematic bias), evaluate the study in the context of related studies and other evidence, and evaluate whether the study can be reasonably considered to have proven its conclusions. To underscore the need for peer review and the danger of overgeneralizing conclusions, two Boston-area medical researchers performed a randomized controlled trial in which they randomly assigned either a parachute or an empty backpack to 23 volunteers who jumped from either a biplane or a helicopter. The study was able to accurately report that parachutes fail to reduce injury compared to empty backpacks. The key context that limited the general applicability of this conclusion was that the aircraft were parked on the ground, and participants had only jumped about two feet.^[87]

Advantages

RCTs are considered to be the most reliable form of scientific evidence in the hierarchy of evidence that influences healthcare policy and practice because RCTs reduce spurious causality and bias. Results of RCTs may be combined in systematic reviews which are increasingly being used in the conduct of evidence-based practice. Some examples of scientific organizations' considering RCTs or systematic reviews of RCTs to be the highest-quality evidence available are:

• As of 1998, the National Health and Medical Research Council of Australia designated "Level I" evidence as that "obtained from a systematic review of all relevant randomised controlled trials" and "Level II" evidence as that "obtained from at least one properly designed randomised controlled trial."^[88]
• Since at least 2001, in making clinical practice guideline recommendations the United States Preventive Services Task Force has considered both a study's design and its internal validity as indicators of its quality.^[89] It has recognized "evidence obtained from at least one properly randomized controlled trial" with good internal validity (i.e., a rating of "I-good") as the highest quality evidence available to it.^[89]
• The GRADE Working Group concluded in 2008 that "randomised trials without important limitations constitute high quality evidence."^[90]
• For issues involving "Therapy/Prevention, Aetiology/Harm", the Oxford Centre for Evidence-based Medicine as of 2011 defined "Level 1a" evidence as a systematic review of RCTs that are consistent with each other, and "Level 1b" evidence as an "individual RCT (with narrow Confidence Interval)."^[91]

Notable RCTs with unexpected results that contributed to changes in clinical practice include:

• After Food and Drug Administration approval, the antiarrhythmic agents flecainide and encainide came to market in 1986 and 1987 respectively.^[92] The non-randomized studies concerning the drugs were characterized as "glowing",^[93] and their sales increased to a combined total of approximately 165,000 prescriptions per month in early 1989.^[92] In that year, however, a preliminary report of an RCT concluded that the two drugs increased mortality.^[94] Sales of the drugs then decreased.^[92]
• Prior to 2002, based on observational studies, it was routine for physicians to prescribe hormone replacement therapy for post-menopausal women to prevent myocardial infarction.^[93] In 2002 and 2004, however, published RCTs from the Women's Health Initiative claimed that women taking hormone replacement therapy with estrogen plus progestin had a higher rate of myocardial infarctions than women on a placebo, and that estrogen-only hormone replacement therapy caused no reduction in the incidence of coronary heart disease.^{[70] [95]} Possible explanations for the discrepancy between the observational studies and the RCTs involved differences in methodology, in the hormone regimens used, and in the populations studied.^{[96] [97]} The use of hormone replacement therapy decreased after publication of the RCTs.^[98]

Disadvantages

Many papers discuss the disadvantages of RCTs.^{[99] [100] [101]} Among the most frequently cited drawbacks are:

Time and costs

RCTs can be expensive;^[101] one study found 28 Phase III RCTs funded by the National Institute of Neurological Disorders and Stroke prior to 2000 with a total cost of US$335 million,^[102] for a mean cost of US$12 million per RCT. Nevertheless, the return on investment of RCTs may be high, in that the same study projected that the 28 RCTs produced a "net benefit to society at 10-years" of 46 times the cost of the trials program, based on evaluating a quality-adjusted life year as equal to the prevailing mean per capita gross domestic product.^[102]

The conduct of an RCT takes several years until being published; thus, data is restricted from the medical community for long years and may be of less relevance at time of publication.^[103]

It is costly to maintain RCTs for the years or decades that would be ideal for evaluating some interventions.^{[99] [101]}

Interventions to prevent events that occur only infrequently (e.g., sudden infant death syndrome) and uncommon adverse outcomes (e.g., a rare side effect of a drug) would require RCTs with extremely large sample sizes and may, therefore, best be assessed by observational studies.^[99]

Due to the costs of running RCTs, these usually only inspect one variable or very few variables, rarely reflecting the full picture of a complicated medical situation; whereas the case report, for example, can detail many aspects of the patient's medical situation (e.g. patient history, physical examination, diagnosis, psychosocial aspects, follow up).^[103]

Conflict of interest dangers

A 2011 study done to disclose possible conflicts of interests in underlying research studies used for medical meta-analyses reviewed 29 meta-analyses and found that conflicts of interests in the studies underlying the meta-analyses were rarely disclosed. The 29 meta-analyses included 11 from general medicine journals; 15 from specialty medicine journals, and 3 from the Cochrane Database of Systematic Reviews. The 29 meta-analyses reviewed an aggregate of 509 randomized controlled trials (RCTs). Of these, 318 RCTs reported funding sources with 219 (69%) industry funded. 132 of the 509 RCTs reported author conflict of interest disclosures, with 91 studies (69%) disclosing industry financial ties with one or more authors. The information was, however, seldom reflected in the meta-analyses. Only two (7%) reported RCT funding sources and none reported RCT author-industry ties. The authors concluded "without acknowledgment of COI due to industry funding or author industry financial ties from RCTs included in meta-analyses, readers' understanding and appraisal of the evidence from the meta-analysis may be compromised."^[104]

Some RCTs are fully or partly funded by the health care industry (e.g., the pharmaceutical industry) as opposed to government, nonprofit, or other sources. A systematic review published in 2003 found four 1986–2002 articles comparing industry-sponsored and nonindustry-sponsored RCTs, and in all the articles there was a correlation of industry sponsorship and positive study outcome.^[105] A 2004 study of 1999–2001 RCTs published in leading medical and surgical journals determined that industry-funded RCTs "are more likely to be associated with statistically significant pro-industry findings."^[106] These results have been mirrored in trials in surgery, where although industry funding did not affect the rate of trial discontinuation it was however associated with a lower odds of publication for completed trials.^[107] One possible reason for the pro-industry results in industry-funded published RCTs is publication bias.^[106] Other authors have cited the differing goals of academic and industry sponsored research as contributing to the difference. Commercial sponsors may be more focused on performing trials of drugs that have already shown promise in early stage trials, and on replicating previous positive results to fulfill regulatory requirements for drug approval.^[108]

Ethics

If a disruptive innovation in medical technology is developed, it may be difficult to test this ethically in an RCT if it becomes "obvious" that the control subjects have poorer outcomes—either due to other foregoing testing, or within the initial phase of the RCT itself. Ethically it may be necessary to abort the RCT prematurely, and getting ethics approval (and patient agreement) to withhold the innovation from the control group in future RCTs may not be feasible.

Historical control trials (HCT) exploit the data of previous RCTs to reduce the sample size; however, these approaches are controversial in the scientific community and must be handled with care.^[109]

In social science

Due to the recent emergence of RCTs in social science, the use of RCTs in social sciences is a contested issue. Some writers from a medical or health background have argued that existing research in a range of social science disciplines lacks rigour, and should be improved by greater use of randomized control trials.^[110]

Transport science

Researchers in transport science argue that public spending on programmes such as school travel plans could not be justified unless their efficacy is demonstrated by randomized controlled trials.^[111] Graham-Rowe and colleagues^[112] reviewed 77 evaluations of transport interventions found in the literature, categorising them into 5 "quality levels". They concluded that most of the studies were of low quality and advocated the use of randomized controlled trials wherever possible in future transport research.

Dr. Steve Melia^[113] took issue with these conclusions, arguing that claims about the advantages of RCTs, in establishing causality and avoiding bias, have been exaggerated. He proposed the following eight criteria for the use of RCTs in contexts where interventions must change human behaviour to be effective:

The intervention:

1. Has not been applied to all members of a unique group of people (e.g. the population of a whole country, all employees of a unique organisation etc.)
2. Is applied in a context or setting similar to that which applies to the control group
3. Can be isolated from other activities—and the purpose of the study is to assess this isolated effect
4. Has a short timescale between its implementation and maturity of its effects

And the causal mechanisms:

5. Are either known to the researchers, or else all possible alternatives can be tested
6. Do not involve significant feedback mechanisms between the intervention group and external environments
7. Have a stable and predictable relationship to exogenous factors
8. Would act in the same way if the control group and intervention group were reversed

Criminology

A 2005 review found 83 randomized experiments in criminology published in 1982–2004, compared with only 35 published in 1957–1981.^[114] The authors classified the studies they found into five categories: "policing", "prevention", "corrections", "court", and "community".^[114] Focusing only on offending behavior programs, Hollin (2008) argued that RCTs may be difficult to implement (e.g., if an RCT required "passing sentences that would randomly assign offenders to programmes") and therefore that experiments with quasi-experimental design are still necessary.^[115]

Education

RCTs have been used in evaluating a number of educational interventions. Between 1980 and 2016, over 1,000 reports of RCTs have been published.^[116] For example, a 2009 study randomized 260 elementary school teachers' classrooms to receive or not receive a program of behavioral screening, classroom intervention, and parent training, and then measured the behavioral and academic performance of their students.^[117] Another 2009 study randomized classrooms for 678 first-grade children to receive a classroom-centered intervention, a parent-centered intervention, or no intervention, and then followed their academic outcomes through age 19.^[118]

Criticism

A 2018 review of the 10 most cited randomised controlled trials noted poor distribution of background traits, difficulties with blinding, and discussed other assumptions and biases inherent in randomised controlled trials. These include the "unique time period assessment bias", the "background traits remain constant assumption", the "average treatment effects limitation", the "simple treatment at the individual level limitation", the "all preconditions are fully met assumption", the "quantitative variable limitation" and the "placebo only or conventional treatment only limitation".^[119]

See also

• Drug development
• Hypothesis testing
• Impact evaluation
• Jadad scale
• Pipeline planning
• Patient and public involvement
• Observational study
• Blinded experiment
• Statistical inference

Further reading

• Web site: Bland M . Directory of randomisation software and services. . University of York . 19 March 2008 .
• Book: Evans I, Thornton H, Chalmers I . Testing treatments: better research for better health care. . London . Pinter & Martin . 2010 . 978-1-905177-35-6 . https://web.archive.org/web/20101225093331/http://www.jameslindlibrary.org/pdf/testing-treatments.pdf . 2010-12-25 .
• Book: Gelband H . The impact of randomized clinical trials on health policy and medical practice: background paper. . https://web.archive.org/web/20160303191654/http://www.fas.org/ota/reports/8310.pdf . 2016-03-03 . Washington, DC . U.S. Congress, Office of Technology Assessment . 1983 . OTA-BP-H-22 .
• Web site: REFLECT (Reporting guidElines For randomized controLled trials for livEstoCk and food safeTy) Statement .
• Web site: Wathen JK, Cook JD . Power and bias in adaptively randomized clinical trials. . M.D. Anderson Cancer Center, University of Texas . 12 July 2006 .

Notes and References

1. Chalmers TC, Smith H, Blackburn B, Silverman B, Schroeder B, Reitman D, Ambroz A . A method for assessing the quality of a randomized control trial . Controlled Clinical Trials . 2 . 1 . 31–49 . May 1981 . 7261638 . 10.1016/0197-2456(81)90056-8 .
2. Web site: What Are Clinical Trials and Studies? . National Institute on Aging, US National Institutes of Health . 9 March 2024 . 22 March 2023.
3. Web site: What Are the Different Types of Clinical Research? . US Food and Drug Administration . 9 March 2024 . 4 January 2018.
4. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG . CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials . BMJ . 340 . c869 . March 2010 . 20332511 . 2844943 . 10.1136/bmj.c869 .
5. Hannan EL . Randomized clinical trials and observational studies: guidelines for assessing respective strengths and limitations . JACC. Cardiovascular Interventions . 1 . 3 . 211–217 . June 2008 . 19463302 . 10.1016/j.jcin.2008.01.008 . free .
6. Ranjith G . Interferon-alpha-induced depression: when a randomized trial is not a randomized controlled trial . Psychotherapy and Psychosomatics . 74 . 6 . 387; author reply 387-387; author reply 388 . 2005 . 16244516 . 10.1159/000087787 . 143644933 .
7. Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV, Mantel N, McPherson K, Peto J, Smith PG . Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design . British Journal of Cancer . 34 . 6 . 585–612 . December 1976 . 795448 . 2025229 . 10.1038/bjc.1976.220 .
8. Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV, Mantel N, McPherson K, Peto J, Smith PG . Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. analysis and examples . British Journal of Cancer . 35 . 1 . 1–39 . January 1977 . 831755 . 2025310 . 10.1038/bjc.1977.1 .
9. Wollert KC, Meyer GP, Lotz J, Ringes-Lichtenberg S, Lippolt P, Breidenbach C, Fichtner S, Korte T, Hornig B, Messinger D, Arseniev L, Hertenstein B, Ganser A, Drexler H . Intracoronary autologous bone-marrow cell transfer after myocardial infarction: the BOOST randomised controlled clinical trial . Lancet . 364 . 9429 . 141–148 . 2004 . 15246726 . 10.1016/S0140-6736(04)16626-9 . 24361586 .
10. Dunn PM . James Lind (1716-94) of Edinburgh and the treatment of scurvy . Archives of Disease in Childhood. Fetal and Neonatal Edition . 76 . 1 . F64–F65 . January 1997 . 9059193 . 1720613 . 10.1136/fn.76.1.f64 .
11. Daston L . Scientific Error and the Ethos of Belief. Social Research. 72. 1. 2005. 18. 10.1353/sor.2005.0016 . 141036212 .
12. Rivers WH, Webber HN . The action of caffeine on the capacity for muscular work . The Journal of Physiology . 36 . 1 . 33–47 . August 1907 . 16992882 . 1533733 . 10.1113/jphysiol.1907.sp001215 .
13. Peirce CS, Jastrow J. 1885. On Small Differences in Sensation. Memoirs of the National Academy of Sciences. 3. 73–83. http://psychclassics.yorku.ca/Peirce/small-diffs.htm
14. 10.1086/354775. Hacking I . Ian Hacking . Telepathy: Origins of Randomization in Experimental Design. Isis. A Special Issue on Artifact and Experiment. 79. 3. September 1988 . 427–451. 234674. 1013489. 52201011.
15. 10.1086/444032. Stigler SM . A Historical View of Statistical Concepts in Psychology and Educational Research. American Journal of Education. 101. 1. November 1992. 60–70. 143685203. Stephen M. Stigler.
16. Dehue T . Deception, efficiency, and random groups. Psychology and the gradual origination of the random group design . Isis; an International Review Devoted to the History of Science and Its Cultural Influences . 88 . 4 . 653–673 . December 1997 . 9519574 . 10.1086/383850 . 23526321 .
17. Woodworth RS, ThorndikeEL. The influence of improvement in one mental function upon the efficiency of other functions (I). Psychological Review. 8. 3. 1901. 247.
18. Coover JE, Angell F . 1907 . General Practice Effect of Special Exercise . The American Journal of Psychology . 18 . 3 . 328–340 . 10.2307/1412596 . 0002-9556.
19. Dehue T . 2000 . From deception trials to control reagents: The introduction of the control group about a century ago. . American Psychologist . en . 55 . 2 . 264–268 . 10.1037/0003-066X.55.2.264 . 1935-990X . https://web.archive.org/web/20240712172031/https://www.rug.nl/staff/trudy.dehue/11%20deception%20trials.pdf . 12 Jul 2024.
20. Neyman, Jerzy. 1923 [1990]. "On the Application of Probability Theory to AgriculturalExperiments. Essay on Principles. Section 9." Statistical Science 5 (4): 465–472. Trans. Dorota M. Dabrowska and Terence P. Speed.
21. According to Denis Conniffe:News: R. A. Fisher and the development of statistics—a view in his centenary year. Conniffe D . . Dublin. Journal of the Statistical and Social Inquiry Society of Ireland. XXVI. 3. 1990–1991. 87 . 2262/2764. 0081-4776 . Book: Mann HB . Henry Mann. Analysis and design of experiments: Analysis of variance and analysis of variance designs. Dover Publications . New York . 1949 . 32177.
22. ((Streptomycin in Tuberculosis Trials Committee)) . STREPTOMYCIN treatment of pulmonary tuberculosis . British Medical Journal . 2 . 4582 . 769–782 . October 1948 . 18890300 . 2091872 . 10.1136/bmj.2.4582.769 .
23. News: Landmark study made research resistant to bias . Brown D . . 1998-11-02 .
24. Shikata S, Nakayama T, Noguchi Y, Taji Y, Yamagishi H . Comparison of effects in randomized controlled trials with observational studies in digestive surgery . Annals of Surgery . 244 . 5 . 668–676 . November 2006 . 17060757 . 1856609 . 10.1097/01.sla.0000225356.04304.bc .
25. Stolberg HO, Norman G, Trop I . Randomized controlled trials . AJR. American Journal of Roentgenology . 183 . 6 . 1539–1544 . December 2004 . 15547188 . 10.2214/ajr.183.6.01831539 . 5376391 .
26. Web site: Ferry G . Peter Sleight Obituary . The Guardian . 2 November 2020 . 3 November 2020.
27. Meldrum ML . A brief history of the randomized controlled trial. From oranges and lemons to the gold standard . Hematology/Oncology Clinics of North America . 14 . 4 . 745–60, vii . August 2000 . 10949771 . 10.1016/S0889-8588(05)70309-9 .
28. Freedman B . Equipoise and the ethics of clinical research . The New England Journal of Medicine . 317 . 3 . 141–145 . July 1987 . 3600702 . 10.1056/NEJM198707163170304 .
29. Gifford F . Community-equipoise and the ethics of randomized clinical trials . Bioethics . 9 . 2 . 127–148 . April 1995 . 11653056 . 10.1111/j.1467-8519.1995.tb00306.x .
30. Edwards SJ, Lilford RJ, Hewison J . The ethics of randomised controlled trials from the perspectives of patients, the public, and healthcare professionals . BMJ . 317 . 7167 . 1209–1212 . October 1998 . 9794861 . 1114158 . 10.1136/bmj.317.7167.1209 .
31. Zelen M . A new design for randomized clinical trials . The New England Journal of Medicine . 300 . 22 . 1242–1245 . May 1979 . 431682 . 10.1056/NEJM197905313002203 .
32. Torgerson DJ, Roland M . What is Zelen's design? . BMJ . 316 . 7131 . 606 . February 1998 . 9518917 . 1112637 . 10.1136/bmj.316.7131.606 .
33. Appelbaum PS, Roth LH, Lidz C . The therapeutic misconception: informed consent in psychiatric research . International Journal of Law and Psychiatry . 5 . 3–4 . 319–329 . 1982 . 6135666 . 10.1016/0160-2527(82)90026-7 .
34. Henderson GE, Churchill LR, Davis AM, Easter MM, Grady C, Joffe S, Kass N, King NM, Lidz CW, Miller FG, Nelson DK, Peppercorn J, Rothschild BB, Sankar P, Wilfond BS, Zimmer CR . Clinical trials and medical care: defining the therapeutic misconception . PLOS Medicine . 4 . 11 . e324 . November 2007 . 18044980 . 2082641 . 10.1371/journal.pmed.0040324 . free .
35. Jain SL . The mortality effect: counting the dead in the cancer trial . . 21 . 1 . 89–117 . 2010 . 10.1215/08992363-2009-017 . 143641293 . https://web.archive.org/web/20200220002207/https://pdfs.semanticscholar.org/aea1/45d2ff3b9c36b283cd9ca8cb61b839ef6993.pdf . 2020-02-20 .
36. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJ, Schroeder TV, Sox HC, Van Der Weyden MB . Clinical trial registration: a statement from the International Committee of Medical Journal Editors . The New England Journal of Medicine . 351 . 12 . 1250–1251 . September 2004 . 15356289 . 10.1056/NEJMe048225 . free .
37. Law MR, Kawasumi Y, Morgan SG . Despite law, fewer than one in eight completed studies of drugs and biologics are reported on time on ClinicalTrials.gov . Health Affairs . 30 . 12 . 2338–2345 . December 2011 . 22147862 . 10.1377/hlthaff.2011.0172 . free .
38. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P . Comparison of registered and published primary outcomes in randomized controlled trials . JAMA . 302 . 9 . 977–984 . September 2009 . 19724045 . 10.1001/jama.2009.1242 . free .
39. Bhaumik S, Biswas T . Editorial policies of MEDLINE indexed Indian journals on clinical trial registration . Indian Pediatrics . 50 . 3 . 339–340 . March 2013 . 23680610 . 10.1007/s13312-013-0092-2 . 40317464 .
40. Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG . The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed . BMJ . 340 . c723 . March 2010 . 20332510 . 2844941 . 10.1136/bmj.c723 .
41. Kaiser J, Niesen W, Probst P, Bruckner T, Doerr-Harim C, Strobel O, Knebel P, Diener MK, Mihaljevic AL, Büchler MW, Hackert T . Abdominal drainage versus no drainage after distal pancreatectomy: study protocol for a randomized controlled trial . Trials . 20 . 1 . 332 . June 2019 . 31174583 . 6555976 . 10.1186/s13063-019-3442-0 . free .
42. Farag SM, Mohammed MO, El-Sobky TA, ElKadery NA, ElZohiery AK . Botulinum Toxin A Injection in Treatment of Upper Limb Spasticity in Children with Cerebral Palsy: A Systematic Review of Randomized Controlled Trials . JBJS Reviews . 8 . 3 . e0119 . March 2020 . 32224633 . 7161716 . 10.2106/JBJS.RVW.19.00119 . free .
43. Book: Jones B, Kenward MG . Design and Analysis of Cross-Over Trials . Second. London: Chapman and Hall . 2003 .
44. Book: Vonesh EF, Chinchilli VG . Crossover Experiments. 111–202. Linear and Nonlinear Models for the Analysis of Repeated Measurements . London: Chapman and Hall . 1997 .
45. Gall S, Adams L, Joubert N, Ludyga S, Müller I, Nqweniso S, Pühse U, du Randt R, Seelig H, Smith D, Steinmann P, Utzinger J, Walter C, Gerber M . Effect of a 20-week physical activity intervention on selective attention and academic performance in children living in disadvantaged neighborhoods: A cluster randomized control trial . PLOS ONE . 13 . 11 . e0206908 . 8 November 2018 . 30408073 . 6224098 . 10.1371/journal.pone.0206908 . free . 2018PLoSO..1306908G .
46. Gladstone MJ, Chandna J, Kandawasvika G, Ntozini R, Majo FD, Tavengwa NV, Mbuya MN, Mangwadu GT, Chigumira A, Chasokela CM, Moulton LH, Stoltzfus RJ, Humphrey JH, Prendergast AJ . Independent and combined effects of improved water, sanitation, and hygiene (WASH) and improved complementary feeding on early neurodevelopment among children born to HIV-negative mothers in rural Zimbabwe: Substudy of a cluster-randomized trial . PLOS Medicine . 16 . 3 . e1002766 . March 2019 . 30897095 . 6428259 . 10.1371/journal.pmed.1002766 . free .
47. Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D . Improving the reporting of pragmatic trials: an extension of the CONSORT statement . BMJ . 337 . a2390 . November 2008 . 19001484 . 3266844 . 10.1136/bmj.a2390 .
48. Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ . Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement . JAMA . 295 . 10 . 1152–1160 . March 2006 . 16522836 . 10.1001/jama.295.10.1152 .
49. Schulz KF, Grimes DA . Generation of allocation sequences in randomised trials: chance, not choice . Lancet . 359 . 9305 . 515–519 . February 2002 . 11853818 . 10.1016/S0140-6736(02)07683-3 . 291300 .
50. Howick J, Mebius A . In search of justification for the unpredictability paradox . Trials . 15 . 480 . December 2014 . 25490908 . 4295227 . 10.1186/1745-6215-15-480 . free .
51. Lachin JM . Statistical properties of randomization in clinical trials . Controlled Clinical Trials . 9 . 4 . 289–311 . December 1988 . 3060315 . 10.1016/0197-2456(88)90045-1 .
52. Web site: Rosenberger J . STAT 503 - Design of Experiments . Pennsylvania State University. 24 September 2012.
53. Avins AL . Can unequal be more fair? Ethics, subject allocation, and randomised clinical trials . Journal of Medical Ethics . 24 . 6 . 401–408 . December 1998 . 9873981 . 479141 . 10.1136/jme.24.6.401 .
54. Buyse ME . Analysis of clinical trial outcomes: some comments on subgroup analyses . Controlled Clinical Trials . 10 . 4 Suppl . 187S–194S . December 1989 . 2605967 . 10.1016/0197-2456(89)90057-3 .
55. Lachin JM, Matts JP, Wei LJ . Randomization in clinical trials: conclusions and recommendations . Controlled Clinical Trials . 9 . 4 . 365–374 . December 1988 . 3203526 . 10.1016/0197-2456(88)90049-9 . free . 2027.42/27041 .
56. Rosenberger WF, Lachin JM . The use of response-adaptive designs in clinical trials . Controlled Clinical Trials . 14 . 6 . 471–484 . December 1993 . 8119063 . 10.1016/0197-2456(93)90028-C .
57. Forder PM, Gebski VJ, Keech AC . Allocation concealment and blinding: when ignorance is bliss . The Medical Journal of Australia . 182 . 2 . 87–89 . January 2005 . 15651970 . 10.5694/j.1326-5377.2005.tb06584.x . 202149 .
58. Schulz KF, Grimes DA . Allocation concealment in randomised trials: defending against deciphering . Lancet . 359 . 9306 . 614–618 . February 2002 . 11867132 . 10.1016/S0140-6736(02)07750-4 . 12902486 . https://wayback.archive-it.org/all/20120911210240/http://www.who.int/entity/rhl/LANCET_614-618.pdf . September 11, 2012 .
59. Pildal J, Chan AW, Hróbjartsson A, Forfang E, Altman DG, Gøtzsche PC . Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study . BMJ . 330 . 7499 . 1049 . May 2005 . 15817527 . 557221 . 10.1136/bmj.38414.422650.8F .
60. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA . Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study . BMJ . 336 . 7644 . 601–605 . March 2008 . 18316340 . 2267990 . 10.1136/bmj.39465.451748.AD .
61. Book: Princeton University Press . 978-0-691-15924-9 . Glennerster R, Kudzai T . Running randomized evaluations: a practical guide . Princeton . 2013 . "Chapter 6" . 10.2307/j.ctt4cgd52 . j.ctt4cgd52 .
62. Devereaux PJ, Manns BJ, Ghali WA, Quan H, Lacchetti C, Montori VM, Bhandari M, Guyatt GH . Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials . JAMA . 285 . 15 . 2000–2003 . April 2001 . 11308438 . 10.1001/jama.285.15.2000 . free .
63. Haahr MT, Hróbjartsson A . Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors . Clinical Trials . 3 . 4 . 360–365 . 2006 . 17060210 . 10.1177/1740774506069153 . 23818514 .
64. Marson AG, Al-Kharusi AM, Alwaidh M, Appleton R, Baker GA, Chadwick DW, Cramp C, Cockerell OC, Cooper PN, Doughty J, Eaton B, Gamble C, Goulding PJ, Howell SJ, Hughes A, Jackson M, Jacoby A, Kellett M, Lawson GR, Leach JP, Nicolaides P, Roberts R, Shackley P, Shen J, Smith DF, Smith PE, Smith CT, Vanoli A, Williamson PR. The SANAD study of effectiveness of valproate, lamotrigine, or topiramate for generalised and unclassifiable epilepsy: an unblinded randomised controlled trial . Lancet . 369 . 9566 . 1016–1026 . March 2007 . 17382828 . 2039891 . 10.1016/S0140-6736(07)60461-9 .
65. Chan R, Hemeryck L, O'Regan M, Clancy L, Feely J . Oral versus intravenous antibiotics for community acquired lower respiratory tract infection in a general hospital: open, randomised controlled trial . BMJ . 310 . 6991 . 1360–1362 . May 1995 . 7787537 . 2549744 . 10.1136/bmj.310.6991.1360 .
66. Fukase K, Kato M, Kikuchi S, Inoue K, Uemura N, Okamoto S, Terao S, Amagai K, Hayashi S, Asaka M. Effect of eradication of Helicobacter pylori on incidence of metachronous gastric carcinoma after endoscopic resection of early gastric cancer: an open-label, randomised controlled trial . Lancet . 372 . 9636 . 392–397 . August 2008 . 18675689 . 10.1016/S0140-6736(08)61159-9 . free . 13741892 . 2115/34681 .
67. Noseworthy JH, Ebers GC, Vandervoort MK, Farquhar RE, Yetisir E, Roberts R . The impact of blinding on the results of a randomized, placebo-controlled multiple sclerosis clinical trial . Neurology . 44 . 1 . 16–20 . January 1994 . 8290055 . 10.1212/wnl.44.1.16 . 2010-03-25 . 2663997 . https://web.archive.org/web/20050510212611/http://www.neurology.org/cgi/content/abstract/44/1/16 . 2005-05-10 . John H. Noseworthy .
68. Manns MP, McHutchison JG, Gordon SC, Rustgi VK, Shiffman M, Reindollar R, Goodman ZD, Koury K, Ling M, Albrecht JK . Peginterferon alfa-2b plus ribavirin compared with interferon alfa-2b plus ribavirin for initial treatment of chronic hepatitis C: a randomised trial . Lancet . 358 . 9286 . 958–965 . September 2001 . 11583749 . 10.1016/S0140-6736(01)06102-5 . 14583372 .
69. Schwartz GG, Olsson AG, Ezekowitz MD, Ganz P, Oliver MF, Waters D, Zeiher A, Chaitman BR, Leslie S, Stern T . Effects of atorvastatin on early recurrent ischemic events in acute coronary syndromes: the MIRACL study: a randomized controlled trial . JAMA . 285 . 13 . 1711–1718 . April 2001 . 11277825 . 10.1001/jama.285.13.1711 . free .
70. Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, Jackson RD, Beresford SA, Howard BV, Johnson KC, Kotchen JM, Ockene J . Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women's Health Initiative randomized controlled trial . JAMA . 288 . 3 . 321–333 . July 2002 . 12117397 . 10.1001/jama.288.3.321 . 20149703 . free .
71. Hollis S, Campbell F . What is meant by intention to treat analysis? Survey of published randomised controlled trials . BMJ . 319 . 7211 . 670–674 . September 1999 . 10480822 . 28218 . 10.1136/bmj.319.7211.670 .
72. Web site: Welcome to the CONSORT statement Website . CONSORT Group . 2010-03-29 . 2019-05-09 . https://web.archive.org/web/20190509205002/http://www.consort-statement.org/ .
73. Campbell MK, Piaggio G, Elbourne DR, Altman DG . Consort 2010 statement: extension to cluster randomised trials . BMJ . 345 . e5661 . September 2012 . 22951546 . 10.1136/bmj.e5661 . free . free . 2164/2742 .
74. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P . Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration . Annals of Internal Medicine . 148 . 4 . 295–309 . February 2008 . 18283207 . 10.7326/0003-4819-148-4-200802190-00008 . free .
75. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P . Methods and processes of the CONSORT Group: example of an extension for trials assessing nonpharmacologic treatments . Annals of Internal Medicine . 148 . 4 . W60–W66 . February 2008 . 18283201 . 10.7326/0003-4819-148-4-200802190-00008-w1 . free .
76. Manyara AM, Davies P, Stewart D, Weir CJ, Young AE, Blazeby J, Butcher NJ, Bujkiewicz S, Chan AW, Dawoud D, Offringa M, Ouwens M, Hróbjartssson A, Amstutz A, Bertolaccini L, Bruno VD, Devane D, Faria CD, Gilbert PB, Harris R, Lassere M, Marinelli L, Markham S, Powers JH 3rd, Rezaei Y, Richert L, Schwendicke F, Tereshchenko LG, Thoma A, Turan A, Worrall A, Christensen R, Collins GS, Ross JS, Taylor RS, Ciani O . Reporting of surrogate endpoints in randomised controlled trial reports (CONSORT-Surrogate): extension checklist with explanation and elaboration . BMJ . 386 . e078524 . July 2024 . 38981645 . 10.1136/bmj-2023-078524 . free . 11231881 .
77. Benson K, Hartz AJ . A comparison of observational studies and randomized, controlled trials . The New England Journal of Medicine . 342 . 25 . 1878–1886 . June 2000 . 10861324 . 10.1056/NEJM200006223422506 . free .
78. Concato J, Shah N, Horwitz RI . Randomized, controlled trials, observational studies, and the hierarchy of research designs . The New England Journal of Medicine . 342 . 25 . 1887–1892 . June 2000 . 10861325 . 1557642 . 10.1056/NEJM200006223422507 .
79. Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, Contopoulos-Ioannidis DG, Lau J . Comparison of evidence of treatment effects in randomized and nonrandomized studies . JAMA . 286 . 7 . 821–830 . August 2001 . 11497536 . 10.1001/jama.286.7.821 . 10.1.1.590.2854 .
80. Toews . Ingrid . Anglemyer . Andrew . Nyirenda . John Lz . Alsaid . Dima . Balduzzi . Sara . Grummich . Kathrin . Schwingshackl . Lukas . Bero . Lisa . 2024-01-04 . Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study . The Cochrane Database of Systematic Reviews . 1 . 1 . MR000034 . 10.1002/14651858.MR000034.pub3 . 1469-493X . 10765475 . 38174786. January 4, 2025 .
81. Vandenbroucke JP . Observational research, randomised trials, and two views of medical science . PLOS Medicine . 5 . 3 . e67 . March 2008 . 18336067 . 2265762 . 10.1371/journal.pmed.0050067 . free .
82. Glasziou P, Chalmers I, Rawlins M, McCulloch P . When are randomised trials unnecessary? Picking signal from noise . BMJ . 334 . 7589 . 349–351 . February 2007 . 17303884 . 1800999 . 10.1136/bmj.39070.527986.68 .
83. Einhorn LH . Curing metastatic testicular cancer . Proceedings of the National Academy of Sciences of the United States of America . 99 . 7 . 4592–4595 . April 2002 . 11904381 . 123692 . 10.1073/pnas.072067999 . Lawrence Einhorn . free .
84. Wittes J . Sample size calculations for randomized controlled trials . Epidemiologic Reviews . 24 . 1 . 39–53 . 2002 . 12119854 . 10.1093/epirev/24.1.39 . free .
85. Freiman JA, Chalmers TC, Smith H, Kuebler RR . The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 "negative" trials . The New England Journal of Medicine . 299 . 13 . 690–694 . September 1978 . 355881 . 10.1056/NEJM197809282991304 .
86. Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P . Reporting of sample size calculation in randomised controlled trials: review . free . BMJ . 338 . b1732 . May 2009 . 19435763 . 2680945 . 10.1136/bmj.b1732 .
87. News: Researchers Show Parachutes Don't Work, But There's A Catch . 22 Dec 2018 . Harris R . NPR . live . https://web.archive.org/web/20240117012339/https://www.npr.org/sections/health-shots/2018/12/22/679083038/researchers-show-parachutes-dont-work-but-there-s-a-catch . Jan 17, 2024 .
88. Book: A guide to the development, implementation and evaluation of clinical practice guidelines . National Health and Medical Research Council . 1998-11-16 . Commonwealth of Australia . Canberra . 978-1-86496-048-8 . 56 . 2010-03-28 . 2017-10-14 . https://web.archive.org/web/20171014045833/https://www.nhmrc.gov.au/_files_nhmrc/file/publications/synopses/cp30.pdf .
89. Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, Atkins D . Current methods of the US Preventive Services Task Force: a review of the process . American Journal of Preventive Medicine . 20 . 3 Suppl . 21–35 . April 2001 . 11306229 . 10.1016/S0749-3797(01)00261-6 . Methods Work Group; Third US Preventive Services Task Force .
90. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ . What is "quality of evidence" and why is it important to clinicians? . BMJ . 336 . 7651 . 995–998 . May 2008 . 18456631 . 2364804 . 10.1136/bmj.39490.551019.BE . GRADE Working Group .
91. Web site: Levels of evidence . Oxford Centre for Evidence-based Medicine . 2011-09-16 . 2012-02-15.
92. Anderson JL, Pratt CM, Waldo AL, Karagounis LA . Impact of the Food and Drug Administration approval of flecainide and encainide on coronary artery disease mortality: putting "Deadly Medicine" to the test . The American Journal of Cardiology . 79 . 1 . 43–47 . January 1997 . 9024734 . 10.1016/S0002-9149(96)00673-X .
93. News: In medicine, evidence can be confusing - deluged with studies, doctors try to sort out what works, what doesn't . Rubin R . USA Today . 2006-10-16 . 2010-03-22.
94. Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction . The New England Journal of Medicine . 321 . 6 . 406–412 . August 1989 . 2473403 . 10.1056/NEJM198908103210629 . Cardiac Arrhythmia Suppression Trial (CAST) Investigators .
95. Anderson GL, Limacher M, Assaf AR, Bassford T, Beresford SA, Black H, Bonds D, Brunner R, Brzyski R, Caan B, Chlebowski R, Curb D, Gass M, Hays J, Heiss G, Hendrix S, Howard BV, Hsia J, Hubbell A, Jackson R, Johnson KC, Judd H, Kotchen JM, Kuller L, LaCroix AZ, Lane D, Langer RD, Lasser N, Lewis CE, Manson J, Margolis K, Ockene J, O'Sullivan MJ, Phillips L, Prentice RL, Ritenbaugh C, Robbins J, Rossouw JE, Sarto G, Stefanick ML, Van Horn L, Wactawski-Wende J, Wallace R, Wassertheil-Smoller S . Effects of conjugated equine estrogen in postmenopausal women with hysterectomy: the Women's Health Initiative randomized controlled trial . JAMA . 291 . 14 . 1701–1712 . April 2004 . 15082697 . 10.1001/jama.291.14.1701 . free .
96. Grodstein F, Clarkson TB, Manson JE . Understanding the divergent data on postmenopausal hormone therapy . The New England Journal of Medicine . 348 . 7 . 645–650 . February 2003 . 12584376 . 10.1056/NEJMsb022365 .
97. Vandenbroucke JP . The HRT controversy: observational studies and RCTs fall in line . Lancet . 373 . 9671 . 1233–1235 . April 2009 . 19362661 . 10.1016/S0140-6736(09)60708-X . 44991220 .
98. Hsu A, Card A, Lin SX, Mota S, Carrasquillo O, Moran A . Changes in postmenopausal hormone replacement therapy use among women with high cardiovascular risk . American Journal of Public Health . 99 . 12 . 2184–2187 . December 2009 . 19833984 . 2775780 . 10.2105/AJPH.2009.159889 .
99. Black N . Why we need observational studies to evaluate the effectiveness of health care . BMJ . 312 . 7040 . 1215–1218 . May 1996 . 8634569 . 2350940 . 10.1136/bmj.312.7040.1215 .
100. Bell SH, Peck LR . Obstacles to and limitations of social experiments: 15 false alarms . Abt Thought Leadership Paper Series . 2012 .
101. Sanson-Fisher RW, Bonevski B, Green LW, D'Este C . Limitations of the randomized controlled trial in evaluating population-based health interventions . American Journal of Preventive Medicine . 33 . 2 . 155–161 . August 2007 . 17673104 . 10.1016/j.amepre.2007.04.007 .
102. Johnston SC, Rootenberg JD, Katrak S, Smith WS, Elkins JS . Effect of a US National Institutes of Health programme of clinical trials on public health and costs . Lancet . 367 . 9519 . 1319–1327 . April 2006 . 16631910 . 10.1016/S0140-6736(06)68578-4 . 41035177 .
103. Yitschaky O, Yitschaky M, Zadik Y . Case report on trial: Do you, Doctor, swear to tell the truth, the whole truth and nothing but the truth? . Journal of Medical Case Reports . 5 . 1 . 179 . May 2011 . 21569508 . 3113995 . 10.1186/1752-1947-5-179 . free .
104. Web site: How Well Do Meta-Analyses Disclose Conflicts of Interests in Underlying Research Studies | The Cochrane Collaboration . Cochrane.org . 2011-08-19 . 2014-12-16 . https://web.archive.org/web/20141216145035/http://www.cochrane.org/news/blog/how-well-do-meta-analyses-disclose-conflicts-interests-underlying-research-studies .
105. Bekelman JE, Li Y, Gross CP . Scope and impact of financial conflicts of interest in biomedical research: a systematic review . JAMA . 289 . 4 . 454–465 . 2003 . 12533125 . 10.1001/jama.289.4.454 .
106. Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S, Mears D, Schemitsch EH, Heels-Ansdell D, Devereaux PJ . Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials . CMAJ . 170 . 4 . 477–480 . February 2004 . 14970094 . 332713 .
107. Chapman SJ, Shelton B, Mahmood H, Fitzgerald JE, Harrison EM, Bhangu A . Discontinuation and non-publication of surgical randomised controlled trials: observational study . BMJ . 349 . g6870 . December 2014 . 25491195 . 4260649 . 10.1136/bmj.g6870 .
108. Ridker PM, Torres J . Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000-2005 . JAMA . 295 . 19 . 2270–2274 . May 2006 . 16705108 . 10.1001/jama.295.19.2270 . free .
109. Zhang S, Cao J, Ahn C . Calculating sample size in trials using historical controls . Clinical Trials . 7 . 4 . 343–353 . August 2010 . 20573638 . 3085081 . 10.1177/1740774510373629 .
110. Deaton A, Cartwright N . Understanding and misunderstanding randomized controlled trials . Social Science & Medicine . 210 . 2–21 . August 2018 . 29331519 . 6019115 . 10.1016/j.socscimed.2017.12.005 . Randomized Controlled Trials and Evidence-based Policy: A Multidisciplinary Dialogue .
111. Rowland D, DiGuiseppi C, Gross M, Afolabi E, Roberts I . Randomised controlled trial of site specific advice on school travel patterns . Archives of Disease in Childhood . 88 . 1 . 8–11 . January 2003 . 12495948 . 1719287 . 10.1136/adc.88.1.8 .
112. Graham-Rowe E, Skippon S, Gardner B, Abraham C . 2011 . Can we reduce car use and, if so, how? A review of available evidence. . Transportation Research Part A: Policy and Practice. 44 . 5 . 401–418. 10.1016/j.tra.2011.02.001 . 2011TRPA...45..401G .
113. Melia S . 2011 . Do Randomised Control Trials Offer a Solution to 'low Quality' Transport Research?' . Transportation Research Part A . Bristol . University of the West of England .
114. Farrington DP, Welsh BC . 2005 . Randomized experiments in criminology: What have we learned in the last two decades? . . 1 . 1 . 9–38 . 10.1007/s11292-004-6460-0 . 145758503 .
115. Hollin CR . 2008 . Evaluating offending behaviour programmes: does only randomization glister? . . 8 . 1 . 89–106 . 10.1177/1748895807085871 . 141222135 .
116. Connolly P, Keenan C, Urbanska K . 2018-07-09. The trials of evidence-based practice in education: a systematic review of randomised controlled trials in education research 1980–2016. Educational Research. 60. 3. en. 276–291. 10.1080/00131881.2018.1493353. 0013-1881. free.
117. Walker HM, Seeley JR, Small J, Severson HH, Graham BA, Feil EG, Serna L, Golly AM, Forness SR . 2009 . A randomized controlled trial of the First Step to Success early intervention. Demonstration of program efficacy outcomes in a diverse, urban school district . . 17 . 4 . 197–212 . 10.1177/1063426609341645 . 144571336 .
118. Bradshaw CP, Zmuda JH, Kellam SG, Ialongo NS . Longitudinal Impact of Two Universal Preventive Interventions in First Grade on Educational Outcomes in High School . Journal of Educational Psychology . 101 . 4 . 926–937 . November 2009 . 23766545 . 3678772 . 10.1037/a0016586 .
119. Krauss A . Why all randomised controlled trials produce biased results . Annals of Medicine . 50 . 4 . 312–322 . June 2018 . 29616838 . 10.1080/07853890.2018.1453233 . free .