Significance Testing with the t-Test

R packages

We’ll use the following R packages in this chapter:

• ggplot2
• Sleuth3
• SMPracticals
• arm

Significance testing: time for t

Over recent years, statisticians have pointed out that many scientists can be overly fixated on p-values. This fixation is argued to have contributed to the reproducibility crisis that we discussed earlier in this course. So far, we’ve preferred an estimation-based approach. Estimates and confidence intervals brings us the advantage of staying closer to the science that we’re actually interested in. Luckily, there are significance tests that we can apply to the linear model coefficients. This session introduces a well-known and effective test of significance—the Student’s t-test.

Student’s t-test and Guinness beer

The Student’s t-test got its name because its inventor, William Sealy Gosset (1876–1937), published it using the pseudonym “Student” (Senn 2008). Gosset was the Head Brewer for Guinness and conducted research on how to improve the brewing of their beers. This research included trials of different combinations of yeast, hops, and barley.

The test uses the t-distribution, which can be thought of as a version of the normal distribution, but for smaller sample sizes. Gosset needed this alternative because he performed many experimental comparisons, sometimes with very small sample sizes.