Statistical Hypothesis Testing

Statistical hypothesis testing

What is a hypothesis?

A hypothesis is a testable statement, which is tested using some investigation or experimentation. We make specific assumptions about the dataset which is called the hypothesis and they are further accepted or rejected using some methods.

Hypothesis testing

Hypothesis testing is a statistical analysis that uses sample data to assess two mutually exclusive theories about the properties of a population. We make a hypothesis about the dataset at hand and ask different questions. Hypothesis testing helps us to find the likelihood of answers to those questions that we ask.

These methods produce output which helps us to further accept or reject the assumption made about the dataset.

Null hypothesis (H0)

The assumption we make initially about the dataset and the statistical test assumes, are called the null hypothesis.

Alternative hypothesis (H1)

The alternative assumption holds the assumption we make after the experimentation fails.

We will show you an example to further clarify the definitions of the above terms.

Type 1 Error

The error happens when we fail to accept the null hypothesis meaning we reject the null hypothesis when it should be accepted. It is also called a false positive. The probability of committing a Type 1 error is called the significance level, and it is denoted by $\alpha$.

Type 2 Error

The error that happens when we fail to reject the null hypothesis, meaning we accept the null hypothesis when it should be rejected. It is also called a false negative. The probability of committing a Type 2 error is denoted by $\beta$.

Degrees of freedom

While calculating a population parameter or a sample statistic, we include information about the number of data samples in the calculation. It is the independent piece of information which is used in calculating the population parameter or a sample statistic. It is denoted by $\nu$, $dof$, or $df$.

Approaches to hypothesis testing

P-value approach

In these types of tests, the hypothesis tests return a p-value, which is compared against a chosen level of significance $\alpha$ to reject or fail to reject the null hypothesis. For-example:

• If p-value $\leq\alpha$, reject the null-hypothesis.

• If p-value $>\alpha$, don’t reject the null hypothesis.

Critical-value approach

In these types of test, the hypothesis tests return a test statistic from the data under consideration which can be interpreted in the context of critical values. A critical value is a value from the sampling distribution of the test statistic after which point the null hypothesis can be rejected i.e

• If Test Statistic $<$ Critical Value, don’t reject the null hypothesis.

• If Test Statistic $\geq$ Critical Value, reject the null hypothesis.

One-tailed Test

The one-tailed test has a critical value on the right side of the distribution for non-symmetrical distributions. The test statistic is compared to the calculated critical-value and the results are inferred as mentioned in the critical value approach.

Two-tailed Test

A two-tailed test has two critical values. There is one on each side of the distribution, which is often assumed to be symmetrical. When using a two-tailed test, a significance level (or $\alpha$) used in the calculation of the critical values must be divided by two. This would be split to give two alpha values of 2.5% on either side of the distribution with an acceptance area in the middle of the distribution of 95%. Rules for acceptance or rejection of the hypothesis will change as seen below.

• If Lower Critical Value $\leq$ Test Statistic $\geq$ Upper Critical Value, don’t reject the null hypothesis.

• If Test Statistic < Lower Critical Value OR Test Statistic > Upper Critical Value, then reject the null hypothesis.

If the distribution of the test statistic is symmetric around a mean of zero, then we can shortcut the check by comparing the absolute (positive) value of the test statistic to the upper critical value.

• If |Test Statistic| $\leq$ Upper Critical Value then don’t reject the null hypothesis.

• If |Test Statistic| $>$ Upper Critical Value then reject the null hypothesis.

T-test

It is a statistical hypothesis test where two independent data samples known to have a Gaussian Distribution, have the same Gaussian Distribution.

Null hypothesis: There is no difference between the sample means.

Alternative hypothesis: There is some difference between the sample means.