Fisher Discriminant Analysis

What is Fisher discriminant analysis (FDA)?

Fisher discriminant analysis (FDA) is a popular machine learning algorithm that aims to find the linear combination of input features so that it separates two or more classes. For example, consider a medical research facility using FDA to distinguish between benign and malignant tumors. Data from various tests are collected, and then the two classes are separated.

The idea is to project the input features into a low-dimension space that maximizes class separability. In general, FDA can be used for dimensionality reduction and classification.

To understand better, let’s consider the case of binary classification, where $X^+ = \{x^+_1, x^+_2, ..., x^+_{N_+}\}$ represents the examples belonging to the positive class and $X^- = \{x^-_1, x^-_2, ..., x^-_{N_-}\}$ represents the examples belonging to the negative class . Here, $x^k_i \in \R^n$ is an $n$-dimensional feature vector.

A projection $\theta \in \R^{n}$ is an $n$-dimensional vector such that it projects the $n$-dimensional feature vector $x^k_i \in \R^n$ to a scalar score $y^k_i \in \R$ .