activation_function

Gain insights into deep learning constructs like LSTM networks and autoencoders. Delve into modeling, prediction strategies, and TensorFlow implementation for rare event prediction in real-life scenarios.

Understanding Deep Learning Application in Rare Event Prediction.png

DLAA.tar.gz

key1

UDL1

UDL2

UDL1-copy

This course aims to provide a practical understanding of the key constructs of deep learning, including Multi-layer Perceptrons, Long-Short-Term Memory (LSTM) networks, convolutional neural networks, and autoencoders, which focus on rare event prediction.

Through this course, you’ll gain hands-on experience developing solutions for rare event prediction. You’ll start by modeling rare events that occur infrequently. Next, you’ll explore machine and deep learning solutions for imbalanced data. You’ll then learn about rare event “prediction,” its statistical foundations, and prediction strategies. Finally, you’ll implement real-life prediction models using TensorFlow, a crucial framework for building and training models.

By the end of this course, you’ll have a strong grasp of advanced machine and deep learning techniques in rare event prediction. These exercises will not only help you solve complex issues but also become a firm footing for further study and innovation in this field.

Understanding Deep Learning Applications in Rare Event Prediction

# Custom activation
Activations is an active research field in deep learning and is still at its nascent stage. It’s common among researchers to attempt novel activation ideas. To enable this, custom activation implementation is shown here.
Activations can be defined as a conventional Python function. Their gradient should also be defined and registered to TensorFlow for such definitions.

However, the gradient definition is usually not required if the activation is defined using TensorFlow functions. TensorFlow has derivatives predefined for its built-in functions. Therefore, explicit gradient declaration is not required. Therefore, this approach is simpler and is practically applicable in most activation definitions.

## Thresholded exponential linear unit
Here, a custom activation, thresholded exponential linear unit (`telu`), is defined in the equation below.

$$g(x)=\begin{cases}λx,& \text{if}\space x > 0  \\0,& \text{if}\space −τ≤x≤τ \\λα(\text{exp }x−1),& \text{if}\space x<−τ \end{cases}$$ 

With this activation, weak nodes smaller than $τ$ will be deactivated. The idea behind thresholding small activations is applying regularization directly through the `telu` activation function.
This custom activation is a modification of `selu`. The activation is defined in the code below. 
The message `"Succeeded"` means the weights have been learned properly. 

Learn how to create and implement custom activation functions. 

Getting Started

Rare Event Prediction

Multi-Layer Perceptrons (MLPs)

Long Short-Term Memory (LSTM) Networks

Convolutional Neural Networks (CNNs)

Autoencoders

Conclusion

Custom Activation Functions

Custom activation

Thresholded exponential linear unit