The Pandemic Reminds Us How Science Works

Adults need not worry about the many facts forgotten from science classes. It is more important to remember how science is made. For example, how are treatments found effective? The pandemic has reminded us that medical science is a slow process, sorting through results from clinical trials. The public should have a basic understanding of why there are control groups and random assignment.

It is fair to say scientific knowledge is not common today. Large segments of the country have followed pseudo-scientific reporting on the pandemic. It is time to admit this is not a trivial problem. We not only have people with huge gaps in knowledge, but we also have people hostile to science. Do we have people with power and influence lacking a basic understanding of science? The pandemic tells us yes.

The pandemic also reminds us of the complexity of science. We have taken cures and vaccines for granted. They do not appear because we want them. Scientists work hard to find even partial cures or moderately effective vaccines. Science is not all or nothing. This applies to our field too. Should we move beyond categorical statements that a treatment is empirically validated? How valid is validated?

While science is about precision, public knowledge is about simple, hard facts. It is fine to acknowledge the complexity of a problem, but vague answers irritate everyone. Let us reconsider how we communicate scientific facts. The gold standard for clarity should be helping business leaders make sound, informed decisions. How should such a leader understand treatment effectiveness?

Where are the lines drawn around effective and ineffective treatments? A combination of dotted and solid lines may be needed. For example, we need a bright line dividing us from clearly ineffective treatments. However, science frequently produces good rather than great solutions, along with studies that provide validity to a degree that is not conclusive. We need to convey complexity in simple ways.

Tolerance for pseudo-science must be cut short. The ways that viruses can spread exponentially, live on hard surfaces, and die with simple hand washing are not open to debate. Treatments are not safe and effective unless they have been subjected to randomized controlled trials (RCTs). Anecdotal evidence must be understood as a story and not evidence. A compelling story is still not evidence.

The FDA approves medications for specific diseases or “indications.” Physicians can use their judgment to prescribe for other, “off-label” uses. Other indications can be approved based on RCTs reviewed by the FDA. However, social pressure has more than once preempted the completion of RCTs. The reason for preemption or for prolonged off-label use is not always malicious. Consider cancer treatment, as discussed here in Scientific American:

In the 1990s bone-marrow transplants—in part because of intense lobbying by patient-advocacy groups—became a popular therapy for breast cancer in spite of their complexity, toxicity and cost. About 40,000 women worldwide were treated for a cost as high as $4 billion… A 1999 trial found that transplant therapy conferred “no discernible benefits.” The treatment gave some women acute leukemia, which was “far worse than the cancers they had begun with.”

However, the clarity of the RCT does not settle all discussion. The RCT compares treatment and control groups, and the FDA will approve endless “me too” medications that similarly work better for an indication than a placebo. How does one approved medication compare to another? We have little information in part because drug makers could lose big money as a result of such a comparison.

Research comparing the effectiveness of psychosocial treatments is also vital. Many well-informed consumers undoubtedly believe cognitive behavioral therapy is the most effective form of psychotherapy. Many professionals undoubtedly believe the same thing based on journal articles they have read. In fact, all established forms of therapy today have comparable effectiveness.

Bruce Wampold has analyzed psychotherapy outcomes for decades using meta-analysis since direct comparative studies are few. He has found therapy to be “remarkably efficacious” overall, while no specific therapy emerges as superior. We should all become comfortable with asking “compared to what” in the face of glowing claims for any treatment.

We also must appreciate that all interventions found to be scientifically valid do not share equal empirical support. It matters whether two or 10 research studies have found a treatment to be effective. It also matters that many studies failing to confirm effectiveness have gone unpublished and are ignored as evidence of any sort. We should be comfortable asking how many studies offer support.

One shorthand metric for gauging the empirical support for a treatment is called “Number Needed to Treat” (NNT). It reflects how many people need to be treated to achieve an outcome better than the alternative (e.g., a placebo). Lower numbers are better. The NNT for psychotherapy is 3, for the flu shot 12, and statins 60. The NNT clearly reveals that effective treatments are not all equally effective.

The subjects in research studies also matter. A concrete example can clarify this. The FDA decided in 2013 to cut in half the recommended dose of Ambien for women. They found several women were impaired and overdosing at the original dosage. Biological sex differences matter. Similarly, let us be sure women and minorities are represented in studies on the efficacy of psychosocial treatments.

We are all patients and we can ill afford complacency about healthcare. Let us maintain a firm rejection of interventions falling below the line for scientific evidence. At the same time, we should insist on knowing the degree of empirical support for any validated treatment. The FDA is not infallible. Psychosocial treatments have no FDA. How much should an educated consumer know?

We do not have a clear standard today, but clarification might best start with our own healthcare decisions. My last PCP visit included a discussion of statins. We discussed risk factors, potential side effects, and the NNT for statins. I am not taking a statin today. I am comfortable with my appraisal of the facts. I would even recommend my decision-making process as a norm to follow. Get all the facts.

Ed Jones, PhD, is senior vice president for the Institute for Health and Productivity Management.