Introduction to Bootstrapping and Confidence Interval

In ensemble machine learning, individual models are trained on each bootstrapped sample, and final predictions are based on the collective results of all the models.

There are variations of Bootstrap techniques—three forms that differ primarily in how the population is estimated.

• Nonparametric: It is the most common technique and is resampling with replacement from the original dataset only.

• Semiparametric: It assumes that the population includes other items similar to the observed sample by sampling from a smoothed version of the sample histogram.

• Parametric: It assumes that data comes from a known distribution with unknown parameters.

Let's have a quick overview of the confidence interval.

Confidence interval

In real life, data always has some extent of variations across observations, so confidence intervals are important to build our trust in that variation. We may think about measuring specific lengths nearest to $cm$ and say there is no error or variation in the lengths. However, suppose we reconsider the same measurement nearest to $mm$ or even a micrometer. In that case, we may see significant variations, and here we want a confidence interval to see the stability in our measures within the variations among them.

So, it’s a matter of choice, or a trade-off, how granular we want to go—the reality of variations in the data can’t be ignored. The source of variation could be natural, sampling, explainable, or due to bias. Typically, variations are in a much smaller range across the drug doses. A confidence interval is an indicator of our measurement’s precision. It is also an indicator of how stable the estimate is, which measures how close our measurement will be to the original estimate if we repeat our experiment.

Typically, the confidence interval is computed using its classical formula. However, bootstrapping is an alternative and widely used method for ...