Understanding Deep Learning Applications in Rare Event Prediction/

...

/

Applying Convolutional Networks to Multivariate Time Series

Applying Convolutional Networks to Multivariate Time Series

Discover convolutional networks’ application in predicting rare events within multivariate time series data.

We'll cover the following...

The rare event prediction problem explored in this course is a multivariate time series. Let’s proceed with modeling it with convolutional networks.

Convolution on time series

Before modeling, let’s briefly explore the filters and convolution operation in the context of multivariate time series.

A multivariate time series structure is shown in the image below. It shows an illustrative example in which the x-, y-, and z-axis, show the time, the features, and the features’ values, respectivelymultivariate .

Press + to interact
Multivariate time series
Multivariate time series

The time series in the illustration has three features with rectangular-, upward pulse-, and downward pulse-like movements. The features are placed along with the depth, making them the channels. A filter for such a time series is shown in the illustration below.

Press + to interact
Multivariate time series filter
Multivariate time series filter

The convolution operation between the filter and the time series is shown in the illustration below. As time series has only one spatial axis along time, the convolution sweeps it over time. At each stride, a similarity between the filter and a section of the time series is emitted (not shown in the figure). The convolution variants, such as padding, stride (>1)(>1), dilation, and 1×11 × 1, work similarly along the time axis.

Press + to interact
Illustration of a convolution operation on a multivariate time series
Illustration of a convolution operation on a multivariate time series

Imports and data preparation

Like always, the modeling starts with importing the required libraries, including the user-defined ones.

Press + to interact
import pandas as pd 
import numpy as np
import tensorflow as tf
from tensorflow.keras import optimizers
from tensorflow.keras.models import Model
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Input
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Conv1D
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import MaxPool1D
from tensorflow.keras.layers import AveragePooling1D
from tensorflow.keras.layers import MaxPool2D
from tensorflow.keras.layers import ReLU
from tensorflow.keras.layers import Flatten
from tensorflow.python.keras import backend as K
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from collections import Counter 
import matplotlib.pyplot as plt
import seaborn as sns
# user-defined libraries
import datapreprocessing as dp 
import performancemetrics as pm 
import simpleplots as sp
from numpy.random import seed 
seed (1)
SEED = 123 # used to help randomly select the data points
DATA_SPLIT_PCT = 0.2
from pylab import rcParams 
rcParams['figure.figsize'] = 8, 6 
plt.rcParams.update({'font.size': 22})
print( " Data split percent: ", DATA_SPLIT_PCT )
print( " Random generator seeds: ", SEED )
print( " Size of figures to be plotted later: ", rcParams['figure.figsize'] )

The above code shows data split percent, random generator seed, and size of figures as an output.

The tensor shape of a multivariate time series in a convolutional network is the same as in an LSTMLong Short-Term Memory network. The temporalization procedure discussed in LSTMs is also applied here. The data preprocessing steps for converting categorical features to one-hot encoding and curve-shifting are in the code below.

Press + to interact
df = pd.read_csv("processminer-sheet-break-rare-event-dataset.csv")
df.head(n=5) # visualize the data.
# Hot encoding
hotencoding1 = pd.get_dummies(df['Grade&Bwt'])
hotencoding1 = hotencoding1.add_prefix('grade_') 
hotencoding2 = pd.get_dummies(df['EventPress']) 
hotencoding2 = hotencoding2.add_prefix('eventpress_')
df = df.drop(['Grade&Bwt', 'EventPress'], axis=1)
df = pd.concat([df, hotencoding1 , hotencoding2], axis =1)
# Rename response column name for ease of understanding
df = df.rename(columns={'SheetBreak': 'y'})
# Shift the response column y by 2 rows to do a 4- min ahead prediction.
df = dp.curve_shift(df, shift_by=-2)
# Sort by time and drop the time column.
df['DateTime'] = pd.to_datetime(df.DateTime) 
df = df.sort_values(by='DateTime')
df = df.drop(['DateTime'], axis=1)
# Converts df to numpy array
input_X = df.loc[:, df.columns != 'y'].values 
input_y = df['y'].values
print(df)

The above shows clean data as output, displaying different dataset columns.

Baseline

As ...