Supervised Generative Models

Learn about supervised generative models and linear discriminant analysis.

1D Gaussian example

Classification with generative models has been used for some time. We’ll be discussing an example here which is related to a method called linear discriminant analysis that goes back to a paper by Fisher in 1936. This is the same Fisher who collected the Iris dataset. We will start by outlining the idea in a 1-dimensional Gaussian model, before deriving the more general case with more attributes and classes.

An example of the distribution of an attribute xx for two classes is shown in the figure below. As can be seen, these classes are not fully discriminated by this attribute value, because the attribute values are overlapping. Thus, a good note to remember is that 100100 percent classification is not always possible. However, we can still predict with some high confidence in some cases. A good choice, and indeed the best possible choice, is to predict the blue class on the left for attribute values less than 11 and the red class on the right for attribute values above 11.

Get hands-on with 1400+ tech skills courses.