Feature Space

In both supervised learning and clustering, the term “data point” has been utilized. Specifically, each data point $\bold x$ within the training dataset is represented as a $d$-dimensional vector in $\R^d$ such that $\bold x \in \R^d$. The elements of $\bold x$ are referred to as features. Therefore, $\bold x$ is also referred to as a feature vector.

Feature space

A feature space is a mathematical space that represents the features or attributes of a given dataset. Each observation in the dataset is represented by a vector in the feature space, where each dimension of the vector corresponds to a specific feature or attribute of the observation.

For example, let’s say we have a dataset of cars, and each car is described by its make, model, year, horsepower, and fuel efficiency. The feature space for this dataset would be a five-dimensional space, where each dimension corresponds to one of these features. By analyzing the patterns and relationships among the feature vectors in the feature space, we can gain insights into the underlying structure and characteristics of the dataset.

Subspace example

Consider a dataset of cars $D={(\bold x_1, y_1), (\bold x_2, y_2), \dots, (\bold x_n, y_n)}$, where each $\bold x_i \in \R^d$ ...