Randomized Controlled Trials: The Gold Standard—Really?

In the past few months, we have heard many “experts” rave on about the necessity of having randomized controlled clinical trials (RCTs) to prove one issue or another about whether the management of the COVID-19 virus is worthwhile. As a result, we have been told the virus is not transmitted through the air and then a couple of weeks later hear that the virus is airborne and that the main way of transmission is from person to person. We were told that we do not need masks to protect ourselves and others, only for them to reverse their direction and mandate we all wear masks to save lives. In addition, we have been told certain treatments are not really good because there are no RCTs proving their effectiveness, but there seems to be a significant amount of real-world evidence showing those treatments can work. Finally, we are told that a medication for the treatment of the virus has been found, yet there are no RCTs proving its effectiveness against the virus. What is the deal? 

By definition, an RCT is a medical experiment comparing the effectiveness of 2 or more treatments by allocating subjects to 2 or more groups in a random fashion in an effort to reduce bias in determining the effectiveness of the treatments.¹ In theory, this sounds like a reasonable way to be sure a treatment works, may be better than another therapy, and is safe. Currently, these studies are the gold standard for obtaining evidence about care.² In reality, there are many issues in conducting a truly randomized controlled trial that meets all of the requirements to be one that obtains reliable data. One study of RCTs showed that only 6.8% of the trials were truly RCTs.³ It is important for all of us to remember that the outcome of even randomized trials can be influenced by the inclusion and exclusion criteria. A therapy that is found to be effective in treating diabetic foot ulcers in patients with a hemoglobin A1c (HbA1c) of 7.0 may have no effect at all in the majority of patients who have an HbA1c of 9.0 to 11.0. How many RCTs have been proclaimed to change the way we treat diseases, only to be completely reversed within a few years? When was the last time you saw the results of a large RCT that reported a negative result?⁴ With the cost of development of most products and the cost of trials, a negative trial is rarely reported even if performed. This lack of reporting of negative outcomes discourages investigation of complicated and unusual projects and treatments and, more importantly, it may result in pressure on investigators to falsify their results.⁴ It also may encourage misrepresentation or spinning of the data; “[s]pin can be defined as a specific reporting that fails to faithfully reflect the nature and range of findings and that could affect the impression that the results produce in readers, a way to distort science reporting without actually lying.”⁵ Those who spin data are called spin doctors, and their job is to “modify the perception … to reduce any negative impact or to increase any positive impact on public opinion.”⁶ Many times the spin on clinical trial findings may relate to the financial, intellectual, academic, or political interests of the author(s). Unfortunately, the media is in large part to blame for the marginal or misinformation, with up to 50% of their reports containing spin on the topic.⁷ Next time you see someone making profound and definite statements regarding data from an RCT, you should consider conducting your own research to see if the information is truly worthy of following. The gold standard should be the one you determine for yourself with your own investigation.