"A Trial Showed It." Three Questions That Make a Study Useful to You

"Clinically proven." "A randomized trial showed." "Studies prove it works." These are the strongest words in health news, and for good reason. A randomized trial is one of the fairest tests we have. But a trial answers one specific question, about one group of people. Before it can help you, you need to know what that question was.

What makes a trial fair

In a randomized trial, chance decides who gets the treatment and who doesn't, like a coin toss. That spreads all the other differences between people evenly across the groups: habits, stress, money, genes, even things nobody thought to measure. Many trials also compare against a stand-in treatment and keep people from knowing which group they're in, because hope and expectation are real forces too.

So a trial can tell us something no other study can. Here are three questions to turn its answer into something useful for you.

Question 1: Compared with what?

A treatment can only "work" compared with something else. Look for which one:

- Nothing at all, or a waiting list

- A placebo or sham, a stand-in that looks like the real treatment but leaves out the active part

- The usual care, such as medication or physical therapy

The answer changes everything. In a large German trial of chronic low back pain, about 48 percent of people improved with acupuncture, about 44 percent with sham acupuncture, and about 27 percent with usual care. Compared with the sham, the difference was small. Compared with usual care, both needle groups did far better. Same trial, two very different headlines.

Question 2: Who was in it?

A trial tells us what happened to the people in it. Ask whether they're like you.

- Age and sex

- The health problem they had, and how long they'd had it

- Other conditions, or medications they were taking

- How long the treatment lasted, and how long they were followed

If the trial studied people very different from you, its answer may still be useful, but hold it more loosely.

Question 3: How big was the difference, in real numbers?

Percentages can sound dramatic. Here's a simple made-up example. If a problem affects 2 people out of 100, and a treatment brings that down to 1 out of 100, a headline can say the risk was "cut in half." That's true. It's also one person out of a hundred.

So whenever you see a percentage, ask: "Out of how many?" Try to picture it as people out of 100, or out of 1,000. Then decide whether the difference is big enough to matter in your life.

What fair tests show about this kind of care

When acupuncture, craniosacral therapy and lifestyle care are put to a fair test, here's some of what we see.

- Acupuncture: a pooled analysis of 39 trials and 20,827 people with chronic pain found acupuncture did better than both sham acupuncture and no acupuncture. A year later, most of the benefit was still there.

- Craniosacral therapy: a review of 10 trials with 681 people living with chronic pain found it eased pain more than sham treatment. The trials are small, and more are needed, but the early evidence is promising.

- Lifestyle: in the Diabetes Prevention Program, 3,234 adults at high risk were assigned by chance to a lifestyle program, the drug metformin, or a placebo. The lifestyle program cut new cases of diabetes by 58 percent. The drug cut them by 31 percent.

What a trial can't tell you

Even a great trial reports what happened to the average person in the group. None of us is the average person. A treatment that helped most people may not help you, and one that helped only some may be exactly right for you.

That's why your own record matters. A trial tells you what's likely. Your own careful notes tell you what's actually happening in your body. Together, they make for wise decisions.

A practice for today

Find one health claim that mentions a study or a percentage. Write down three short answers:

1. Compared with what?

2. Who was in it, and are they like me?

3. What's the difference in real numbers, out of 100?

If you can't find an answer, write that down too. Missing answers tell you something.

Words to Carry

- I can ask, "Compared with what?"

- A percentage means little until I know "out of how many."

- The average person is not me.

- Fair tests and my own experience both belong in my choices.

- I keep questioning, and I keep learning.

Tomorrow on Truth in Motion, we ask what the ladder of evidence leaves out, and why not all empiricism is equal.

With you every step of the way,

Dr. Jocelyn

References

Diabetes Prevention Program Research Group. (2002). Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. New England Journal of Medicine, 346(6), 393–403. https://doi.org/10.1056/NEJMoa012512

Haake, M., Müller, H.-H., Schade-Brittinger, C., Basler, H. D., Schäfer, H., Maier, C., Endres, H. G., Trampisch, H. J., & Molsberger, A. (2007). German Acupuncture Trials (GERAC) for chronic low back pain: Randomized, multicenter, blinded, parallel-group trial with 3 groups. Archives of Internal Medicine, 167(17), 1892–1898. https://doi.org/10.1001/archinte.167.17.1892

Haller, H., Lauche, R., Sundberg, T., Dobos, G., & Cramer, H. (2020). Craniosacral therapy for chronic pain: A systematic review and meta-analysis of randomized controlled trials. BMC Musculoskeletal Disorders, 21, 1. https://doi.org/10.1186/s12891-019-3017-y

Vickers, A. J., Vertosick, E. A., Lewith, G., MacPherson, H., Foster, N. E., Sherman, K. J., Irnich, D., Witt, C. M., & Linde, K. (2018). Acupuncture for chronic pain: Update of an individual patient data meta-analysis. The Journal of Pain, 19(5), 455–474. https://doi.org/10.1016/j.jpain.2017.11.005