Naive Bayes

In Machine Learning, Naive Bayes classifiers are a family of simple “probabilistic classifiers”. They are based on applying Bayes’ theorem with strong independence assumptions between the features.

Before we begin to code, let’s learn some theory about Bayes’ theorem.

Bayes’ Theorem

Before we talk more about this classifier and try to code it, we should spend some time on what Bayes’s theorem is.

Bayes’ theorem (alternatively Bayes theorem, Bayes law, or Bayes rule) describes the probability of an event, based on prior knowledge of conditions that might be related to the event.

For example, the historical data shows us that everyone has a probability of 10% to get disease A, which is $P(A)=0.1$. Among the patients in this hospital, they will have probability $P(B)=0.05$ to have symptom B. Meanwhile, patients with disease A with symptom B is 0.1, which is $P(B|A)=0.1$ ...