Some studies used a paired t test and others an unpaired t test. Can I pool them?

(This exchange had two rounds. When something seems odd, I ask for the key details before answering.)

The question: "Some studies report results from a paired t test and others from Student's t test. Can I pool them together, or should I split them into subgroups?"

This question is a little odd; I may need more details from you.

Usually, the outcome we want to compare is analyzed with similar statistical methods across studies. When the methods differ like this, I worry that the studies should not be pooled together at all.

For example, a paired t test is used to compare values in the same patients before and after surgery; Student's t test is used to compare values between different patients divided into two groups. These two kinds of values are not the same, and should not really be pooled.

The follow-up: "I see. I searched for randomized controlled trials, and some older papers are 'randomized, placebo-controlled, cross-over' trials, in which one group of people is compared with itself after a washout period. That is why I asked."

I see. Crossover trials can be included. The best data from a crossover trial come from a paired analysis: the mean of each patient's difference between the two treatments, with its standard error. That effect estimate can be pooled with parallel-group trials using the generic inverse-variance method. It is generally advisable to show crossover and parallel-group trials as separate subgroups.

If a paper reports only the mean and SD for each treatment period, the Cochrane Handbook (Section 23.2.7) describes how to approximate the paired analysis. What you should avoid is analyzing the two periods as if they were two separate groups of patients, because the same patients are in both arms (a unit-of-analysis error).

Using only the first period, before the crossover, is also an option, for example when carry-over is a real concern. But it discards more than half of the information in the trial, the first-period data that happen to be published may be a biased subset, and the choice should be planned in your protocol from the outset.

Reference

• Higgins JPT, Eldridge S, Li T. Chapter 23: Including variants on randomized trials (last updated October 2019), Sections 23.2.4 to 23.2.8. In: Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane, 2024. https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-23