Frequentist vs. Bayesian Statistics

Types of statistics

Frequentist and Bayesian statistics are two approaches to statistical inference used to conclude a population based on sample data.

Frequentist statistics is based on repeated sampling, where the probability of an event is determined by the relative frequency of that event occurring in a large number of independent samples. In this approach, likelihood is considered a long-term relative frequency of an event occurring and is not necessarily associated with any individual event. In frequentist statistics, statistical inference is based on hypothesis testing, where a null hypothesis is assumed to be valid until sufficient evidence is found to reject it in favor of an alternative idea.

Bayesian statistics, on the other hand, is based on subjective probability, where the likelihood of an event is determined by an individual’s belief or degree of confidence in that event occurring. In this approach, probability is considered a measure of an individual’s uncertainty about an event and can be updated as new information becomes available. In Bayesian statistics, statistical inference is based on updating our prior beliefs about an event based on new evidence, using Bayes’ theorem.

Both frequentist and Bayesian statistics have their strengths and weaknesses, and which approach is used depends on the problem being addressed and the goals of the analysis. We'll see them in detail in subsequent sections.

Frequentist statistics

Frequentist statistics is a branch that uses probability theory and assumes that a fixed set of underlying probability distributions can explain all observations. It provides inferences and predictions about a population based on data from a sample. Frequentist statistics aim to infer the population parameters from the sample data. It uses maximum likelihood estimation (MLE)Maximum likelihood estimation estimates the parameters of a population based on the sample data. , confidence intervalsConfidence intervals provide a range of values in which the population parameter is likely to lie., and hypothesis testingHypothesis testing is used to determine whether a given hypothesis is true or false based on the evidence from the sample data. to make inferences about the population. Frequentist statistics has been widely used for many decades and is one of the most popular methods in statistical inference. Frequentist statistics can also calculate the probability of observing a particular importance in the sample data given the population parameters.

Example

Let’s consider an example. The municipal corporation has assigned us a the task of using statistics to suggest a standard door size for all public buildings. In order to do so, we first have to find what the average height of the population in the municipality is. This is shown in the illustration below.