activation_function

Gain insights into deep learning constructs like LSTM networks and autoencoders. Delve into modeling, prediction strategies, and TensorFlow implementation for rare event prediction in real-life scenarios.

Understanding Deep Learning Application in Rare Event Prediction.png

DLAA.tar.gz

key1

UDL1

UDL2

UDL1-copy

This course aims to provide a practical understanding of the key constructs of deep learning, including Multi-layer Perceptrons, Long-Short-Term Memory (LSTM) networks, convolutional neural networks, and autoencoders, which focus on rare event prediction.

Through this course, you’ll gain hands-on experience developing solutions for rare event prediction. You’ll start by modeling rare events that occur infrequently. Next, you’ll explore machine and deep learning solutions for imbalanced data. You’ll then learn about rare event “prediction,” its statistical foundations, and prediction strategies. Finally, you’ll implement real-life prediction models using TensorFlow, a crucial framework for building and training models.

By the end of this course, you’ll have a strong grasp of advanced machine and deep learning techniques in rare event prediction. These exercises will not only help you solve complex issues but also become a firm footing for further study and innovation in this field.

Understanding Deep Learning Applications in Rare Event Prediction

# Baseline network 
Construct a simple sequential baseline model with layer structure as shown below. Note to swap activation and pooling layers.


# Convolution layer 
* `Conv1D` **vs.** `Conv2D` **vs.** `Conv3D`**:** A `Conv‘x’D` is chosen based on the number of spatial axes in the input. Use `Conv1D`, `Conv2D`, and `Conv3D` for inputs with 1, 2, and 3 spatial axes, respectively. Follow the table below for details.



Axes and Channels in Grid-Like Inputs to Convolutional Networks

align-center

* **Kernel:** The `kernel_size` argument is a tuple in which each element represents the size of the kernel along a spatial axis. The size can be taken as:

  $$\frac{\sqrt{\text{spatial axis size}}}{2}$$

* **Filters:** The number of `filters` can be taken as a number from the geometric series of 2 close to $\text{n\_features}/4$.

* **Activation:** Use a `linear` activation. A nonlinear activation will be added after the pooling layer.

* **Padding:** Use `valid` padding in a shallow network. A shallow network is defined as one in which the feature map size does not reduce significantly before the output layer. In deep networks with several convolutional layers, use the `same` padding at least in some of the convolutional layers.

* **Dilation:** Do not use dilation in a baseline model. In deep networks, non-dilated and dilated convolutional layers can be paired.

* **Stride:** Use the default `stride=1` in a baseline model. The stride can be increased to `2` if the input dimension is high. However, in most problems, a stride larger than `2` is not recommended.

# Pooling layer

* `Pool1D` **vs.** `Pool2D` **vs.** `Pool3D`**:** Use the pooling layer consistent with the Conv layer. For example, `Pool1D` with `Conv1D`, and so on.

* **Pooling statistic:** Use the maximum statistic for pooling via
the `MaxPool` layer.

* **Pool size:** The square root of the feature map size (along
the spatial axes) can be taken.

* **Padding:** Follow the same principle as for convolution padding.
Use `valid` padding in a shallow and `same` in a deep network.

# Activation 
Use a nonlinear `relu` activation layer after a pooling layer to follow the structure as shown below:


Learn how to build a baseline convolutional network with a strategic swap of pooling and activation.

Getting Started

Rare Event Prediction

Multi-Layer Perceptrons (MLPs)

Long Short-Term Memory (LSTM) Networks

Convolutional Neural Networks (CNNs)

Autoencoders

Conclusion

Constructing an Optimized Baseline Convolutional Network

Baseline network

Convolution layer