What is local scaling in zero-shot learning (ZSL)?

Local scaling is the process of adjusting the semantic embeddings or representations of classes inside a common space. It aims to scale or normalize the feature vectors of distinct classes in order to better align them in the embedding space, hence improving the recognition process for both seen and unseen classes. Zero-shot learning approaches are employed to reduce discrepancies between various classes’ embeddings by scaling them locally, allowing for more accurate generalization to unseen classes during recognition or classification tasks. This strategy aids in utilizing semantic relationships and similarities across classes that are not explicitly seen during training, enhancing zero-shot learning performance.

How does it work?

Local scaling in zero-shot learning works by adjusting the semantic embeddings or representations of classes in a shared semantic space to facilitate better generalization from seen to unseen classes. Here’s a simplified overview of how it works:

Semantic space representation: Creating a semantic space representation is usually the first step. In zero-shot learning, classes are frequently represented as embeddings in a semantic space. Classes are represented as embeddings based on attributes, word embeddings, or other semantic descriptions at this stage.
Disparity in embeddings: Recognizing and comprehending disparities in embeddings is frequently the next step. It is critical for building successful local scaling approaches to understand why and how the embeddings of seen and unseen classes differ.
Normalization or scaling: In order to fix the observed disparities in embeddings, actual local scaling procedures involving normalization, scaling, or re-weighting are used. Once the disparities are identified, this step can be taken.
Alignment of embeddings: Following the normalization or scaling process, aligning or scaling the embeddings makes classes more similar. This intends to narrow the semantic space gap between known and unknown classes.
Improved generalization: Closing the gap between seen and unseen class embeddings improves generalization. This stage is essential for improving a model’s ability to recognize and categorize unknown classes.
Enhancing classification performance: The preceding steps achieve the overall goal of improving zero-shot learning performance. The model is better suited for accurate categorization since it understands and uses semantic links between classes.

Note: While these steps give a basic sequence, it is important to note that the borders between steps are not necessarily strict, and the particular details might change depending on the zero-shot learning method or model utilized.

Code example

The example below gives the idea of local scaling in zero-shot learning. This example shows how embeddings of known and unknown classes in a common semantic space can be adjusted to improve generalization:

import numpy as np
# Training phase (for seen classes)
num_seen_classes = 5
seen_embeddings = np.random.rand(num_seen_classes, 10)  # 10-dimensional embeddings for each class
# Concatenation
all_embeddings_seen = seen_embeddings
# Normalization for seen classes
normalized_embeddings_seen = (all_embeddings_seen - np.mean(all_embeddings_seen, axis=0)) / np.std(all_embeddings_seen, axis=0)
# Testing phase (for unseen classes)
num_unseen_classes = 3
unseen_embeddings = np.random.rand(num_unseen_classes, 10)  # 10-dimensional embeddings for each class
# Concatenation
all_embeddings_unseen = np.concatenate((seen_embeddings, unseen_embeddings), axis=0)
# Normalization for unseen classes using the mean and std from seen classes
normalized_embeddings_unseen = (all_embeddings_unseen - np.mean(all_embeddings_seen, axis=0)) / np.std(all_embeddings_seen, axis=0)
normalized_seen_embeddings = normalized_embeddings_unseen[:num_seen_classes]
normalized_unseen_embeddings = normalized_embeddings_unseen[num_seen_classes:]
def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
# Cosine similarity calculation for zero-shot learning
unseen_class_index = 0  # Choosing the first unseen class for demonstration
similarities = [cosine_similarity(normalized_unseen_embeddings[unseen_class_index], normalized_seen_embedding)
                for normalized_seen_embedding in normalized_seen_embeddings]
most_similar_seen_class = np.argmax(similarities)
print(f"The most similar seen class to unseen class {unseen_class_index} is class {most_similar_seen_class}")

Code explanation

Let’s break it down:

Line 4–5: Generates random 10-dimensional embeddings for a specified number of seen classes (num_seen_classes = 5).
Line 8: Concatenates the embeddings of seen classes into a single array (all_embeddings_seen).
Line 11: Normalizes the embeddings of seen classes by subtracting the mean and dividing by the standard deviation along each dimension.
Line 14–15: Generates random 10-dimensional embeddings for a specified number of unseen classes (num_unseen_classes = 3).
Line 18: Concatenates the embeddings of both seen and unseen classes into a single array (all_embeddings_unseen).
Line 21: Normalizes the embeddings of unseen classes using the mean and standard deviation calculated from the seen classes.
Line 22–23: Splits the normalized embeddings into those corresponding to seen and unseen classes.
Line 25–26: Defines a function cosine_similarity(a, b) for calculating cosine similarity between two vectors.
Line 29–31: Calculates the cosine similarity between the normalized embeddings of the selected unseen class and each normalized embedding of seen classes.
Line 33–34: Identifies the seen class with the highest cosine similarity to the chosen unseen class and prints the result.
Expected output: The above code generates random embeddings for both seen and unseen classes, normalizes them using local scaling, and then calculates the most similar seen class to a chosen unseen class based on cosine similarity.

Benefits

Local scaling in zero-shot learning provides numerous advantages that help models perform better when dealing with unknown classes:

The model can better generalize from seen to unseen classes by modifying embeddings locally.
Model performance can be hindered by semantic space disparities between seen and unseen classes. Local scaling strategies are utilized to eliminate these discrepancies, allowing the model to better grasp connections between distinct classes.
By emphasizing relevant semantic elements and decreasing noise, scaling embeddings can improve class discrimination. This improvement contributes to better class distinction during classification tasks.
The distributional disparities between seen and unseen classes are frequently addressed by zero-shot learning. Local scaling mitigates the consequences of these distribution changes by more successfully aligning the embeddings.
Local scaling makes it easier to use relationships across classes by aligning embeddings in the semantic space. This allows the model to make better use of similarities and semantic associations during inference.

Limitations

While local scaling in zero-shot learning can offer improvements in handling domain shift and generalization to unseen classes, it also has its limitations:

Local scaling methods rely on semantic representations of classes, such as word embeddings or attribute vectors. However, there might be a semantic gap between the provided semantic representations and the actual visual features of the objects. This can lead to inaccuracies in aligning the representations, resulting in suboptimal local scaling adjustments.
Local scaling methods may risk overfitting the training data, particularly if the training set is small or unbalanced. Adapting the decision boundary for each unseen class based on its similarity to seen classes might lead to overly specific adjustments that do not generalize well to unseen data.
The quality and quantity of semantic information available for each class can vary, which can impact the effectiveness of local scaling. In cases where semantic information is sparse or noisy, the ability to accurately adjust the decision boundary for unseen classes may be limited.
While local scaling aims to address domain shift, it may not completely eliminate the challenge, especially in cases where the domain shift is severe or where there are significant differences between the seen and unseen classes that cannot be adequately captured by semantic representations alone.
Calculating local scaling adjustments for each unseen class can increase the computational complexity, especially when dealing with a large number of unseen classes. This may hinder the scalability of local scaling methods, particularly in real-world applications with a large number of classes.

Free Resources

Learn in-demand tech skills in half the time

PRODUCTS

Mock Interview

New

Courses

Skill Paths

Projects

Assessments