Critical and Observed Values

The t-test

One potentially confusing aspect of the t-test is that it involves two $t-values$. One is the critical value of $t$, which sets the bar for the comparison. This is the minimum $t-value$ needed to achieve a given level of significance ($p = 0.05, 0.01$, and so on). The second is the observed value of $t$, which is calculated by dividing the estimate by its standard error. When the observed value of $t$ is larger than the critical value of $t$, the result is declared statistically significant at that level. Of course, the declaration of significant values at a given level is left over from the early days when computing an exact $p-value$ was necessary. Analysts used tables of critical values as a shortcut.

Although it’s a simple test, in some ways, the t-test brings a lot of extra complexity to the early stages of introductory statistics courses. Gosset needed $t$ as his sample sizes were sometimes very small ($n=4$). However, it isn’t clear how often modern analysts use it, because small sample sizes generally aren’t preferred by statisticians. So far, we’ve followed Gelman and Hill and have taken a refreshingly casual approach that’s often easily able to calculate approximately 95% CIs as $±2$ SEs. This approach is helpful for introductory teaching purposes. But if we use the $t$ distribution instead of assuming that we can apply the normal approximation—what difference does it make?

Critical value of t

The figure below plots the critical value of $t$ (where $p = 0.05$) as a function of sample size. It shows that only when sample sizes get down to single digits that $t$ starts to get much larger than our rough large sample approximation (the red-dashed line).