The paired t-test

In this practical we’ll practice using the paired t-test to analyse an outcome where the observations are paired.


Rationale

The information and instructions given here will be kept brief, as once you understand when to use the paired t-test rather than the independent t-test there’s not much more to know because the interpretation is very similar to the independent t-test. Similarly, the same broad types of limitations apply to the paired t-test as to the independent t-test, so read see that section if you want to understand them.

The paired t-test allows us to more effectively compare whether the mean of an outcome differs between two groups when observations within those two groups are not independent. This is most typically the case when the groups represent measurements taken from individuals or other entities (units of observation) at two points in time. For example, if you were interested in the effect of an intervention on blood pressure you might have measured systolic blood pressure in a group of individuals before the intervention and then repeated the measurements after applying the intervention.

Less clearly you would also have paired observations when collecting outcome measures from pairs of spatially or geographically related entities. For example, if you were interested in HIV rates within villages located along a major highway and were specifically looking at whether villages on the side of the road where traffic is travelling from the capital city have higher HIV rates than villages located on the other side of the road where traffic is travelling to the capital city. You could then compare pairs of villages that are on opposite sides of the road. Clearly though, adjacent villages will likely have correlated HIV rates compared to villages that are further away, and so your observations would be correlated or paired.

In summary, if you think carefully you should be able to work out whether your outcomes are independent or paired based on the study design and subject matter knowledge.

Therefore, if you have paired data and your research question is about whether there are differences between the two groups making up the pairs then you can use the paired t-test. The paired t-test is actually largely just the independent t-test but applied to the differences between the pairs of observations (e.g. the difference between the systolic blood pressure measured before an intervention compared to after an intervention in each individual in a study), rather than applied to two groups of observations. By doing this it accounts for the correlation between paired observations and can make use of this to increase the power and precision of the analysis. Ignoring the paired data and using an independent t-test will simply give more conservative results.

Let’s see how to do the analysis. Note: the way you would store such data for use in a paired t-test is to have two outcome variables: one for each member of the paired observations.

Practice

Scenario

This contains real data from a cluster randomised controlled trial done by Chinese partners (although they are based in Canada). The research question was whether a multi-component complex intervention could reduce inappropriate prescribing of antibiotics by primary care providers to children (aged 2-14) in Chinese primary care facilities. However, this dataset just contains data from the intervention arm before and after the intervention. There are two variables: apr_base and apr_end. apr_base contains the antibiotic prescription rates (actually proportions not true rates) for primary care facilities before the intervention was applied in the intervention arm, and the apr_end contains the antibiotic prescription rates for the same primary care facilities after the intervention had run for six months in the intervention arm. Hence, each row contains data from the same primary care facility, and therefore the outcomes are correlated: they are pre and post measurements from the same facilities.

The antibiotic prescription rates are more specifically the facility-level proportion of prescriptions issued to children (aged 2-14) for upper respiratory tract infections that contain one or more antibiotics. Therefore, broadly speaking these will be inappropriate and a lower rate is more desirable. Therefore, we’ll use a paired t-test to explore the relationship between the antibiotic prescription rate and the intervention period (pre or post intervention) just within the intervention arm. This is equivalent to if we had actually run a uncontrolled before-after study rather than an RCT. Of course, given we had a control arm we would not (and did not) ignore it in reality when we analysed the study, which you can read if you are interested: https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(17)30383-2/fulltext

Exercise 1: use the paired t-test to compare the relationship between the facility-level proportion of appropriately prescribed antibiotics before and after an intervention to reduce inappropriate antibiotic prescribing was applied

  • Load the “AB paired.sav” SPSS dataset.

Step 1: check the assumptions of the paired t-test

1. Continuous outcome

Technically a paired t-test assumes a continuous outcome variable, but as long as the following two assumptions are satisfied it’s fine to use a discrete outcome with a paired t-test.

2. Paired outcome data/observations, but independent observations between pairs

See above for a discussion of how to understand if your data are suitable for the paired t-test. However, the paired t-test also assumes that pairs of observations are independent from each other. For example, if you recorded blood pressure measurements from individuals before and after an intervention the observations would be paired or correlated within individuals, but if individuals were also clustered within family grouping then there would also be correlations between pairs of individuals (as family members would likely have correlated before and after values), and the data would violate this assumption of the paired t-test.

3. Approximately normally distributed differences between paired observations

Compute the difference between each pair of observations and plot them on a histogram to check for approximate normality. In SPSS you can use the Compute Variable tool that we used earlier in the data preparation section to do this easily. You just need to enter the command: “var1 - var2” (without quotes) or vice versa, where var1 and var2 are your paired outcome variables. Or you can also do it in Excel by creating a new column from the difference between your paired outcomes. You should know how to plot a histogram now, but refer back to the “Step 1: check the assumptions of the independent t-test” section earlier for a reminder if needed.

Step 2: run the paired t-test

  • From the main menu go: Analyse > Compare Means > Paired Samples T Test.

  • Add apr_base as Variable1 and apr_end as Variable2 then click OK.

Step 3: understand the results tables and extract the key results

  • The results are largely the same as for the independent t-test, so refer back to the “Step 4: understand the results tables and extract the key results” section in the “Inferential analysis 1: the independent t-test” section to review what the results in the tables mean if you need reminding. However, a few things are a little different. Your descriptive statistics for each group are in the Paired Samples Statistics table, and you get the (presumably) Pearson correlation between the two groups in the Paired Samples Correlations table. Then your inferential results are in the Paired Samples Test table. There are now no equal variances assumed/not assumed version of the results, and we can see the key results are the Mean (i.e. mean difference between the reference group [entered into Variable1] and the comparison group [entered into Variable2], the associated 95% confidence intervals of this mean difference, and if you want it the two-tailed p-value (Sig. (2-tailed)), which tests the hypothesis that the mean difference = 0. You can ignore any “effect size” table.

Step 4: report and interpret the results

For a report we would say something like:

Methods: explain that you used a paired t-test and justify why.

Results: at baseline the mean facility-level antibiotic prescription rate was 0.82. After the six-month post-intervention follow-up the mean facility-level antibiotic prescription rate was 0.4. There was therefore a mean change in the antibiotic prescription rate from baseline to six-month follow-up of 0.42 (95% CI: 0.3, 0.55).

  • Note: take care interpreting the direction of change. If we compared the six-month follow-up group to the baseline group the change would be -0.4, which could be easily mistaken for a reduction when it’s actually an increase. Just be clear on the mean of each group and what comparison is being made, i.e. which group mean is being subtracted from which.

Discussion: interpret the direction and size of the difference in terms of the implications for practice and policy. Is the difference “statistically significant”, i.e. can we make a clear inference that there even is a difference on average between the groups, and if so is it a small difference, a medium difference, a large difference etc in terms of what is being measured and the implications for practice and policy?

Additional exercise: use the paired t-test to analyse whether the number of MDR-TB patients lost to follow-up changed from before to after an intervention to reduce loss to follow-up was applied

Open the simulated “MDR-TB LTFU.sav” dataset. It represents data collected from an uncontrolled pre-post (or before-after) study comparing the number of patients being treated for multiple drug resistant Tuberculosis (MDR-TB) who were lost to follow-up during a month before and after an intervention was implemented that aimed to reduce loss to follow-up. The unit of observation is the health facility, and the dataset contains three variables:

  • facility_id = generic facility ID.

  • ltfu = the number of patients lost to follow-up during the pre or post intervention period.

  • treatment_period = the treatment period (pre or post).

Note: pre-post data are often stored like this in “long” format, with the outcome variable containing repeated measures for each unit of observation and a variable indicating which time period those observations come from. Therefore, you will need to rearrange the dataset into two outcome columns (pre and post) where the observations from each separate facility are on the same row (this should be easy enough to do once you look at the data).

  • Then via the process outlined above use a paired t-test to analyse the relationship between the treatment period (i.e. in theory the effect of the treatment) and the number of MDR-TB patients lost to follow-up per facility.

  • Extract the mean difference and confidence intervals around this estimate.

  • In the “Exercises” folder open the “Exercises.docx” Word document and scroll down to Paired t-test: the relationship between an intervention and MDR-TB patient loss to follow-up.

  • Write a couple of sentences reporting the results of your analysis. Include the basic descriptive statistics: sample size and group outcome means. Also be sure to include sufficient details about the outcome variable and the comparison made, including the type of analysis used, and of course the key inferential results. Round results to one decimal place. You don’t need to explain anything about the study or interpret the clinical or practical importance of the result.

  • Write a sentence or two about the key limitations of this analysis in terms of interpreting the result.

  • Once you’ve completed this compare your reporting to the below example text.

Example results reporting text

Read/hide

Using a paired t-test I analysed the change in the number of MDR-TB patients lost to follow-up during the month before and after an intervention to reduce loss to follow-up in MDR-TB health facilities. Out of a total sample size of 10 health facilities the mean number of patients lost to follow-up before the intervention was 7.9 and after the intervention was 5.3. This represented a mean difference of 2.6 (95% CI: 1, 4.2) more patients lost to follow-up in the month before the intervention was implemented. Therefore, the intervention was associated with a clear and statistically significant reduction in the number of patients lost to follow-up in the month following its implementation compared to the previous month. However, the key limitation of this analysis is that it does not adjust for any other confounding variables, of which there are likely to be many (especially in an uncontrolled pre-post study like this), particularly unmeasured time-varying confounding variables that may have impacted on patient loss to follow-up (such as performance bias [https://catalogofbias.org/biases/performance-bias/]). Therefore, this is likely to represent a biased estimate of the independent relationship between the intervention and the number of patients lost to follow-up.