What are Tensors?
Tensors are multi-dimensional arrays that store numerical data. Tensors can be thought of as a mathematical entity, similar to a generalized matrix, that interacts with other entities in a structure.
Types of tensors
The table below shows the different types of tensors and the names conventionally associated with each type:
Unlike a matrix, the numerical data stored in the tensor shifts given a transformation in one of the entities it interacts with. As such, tensors always obey certain transformation rules.
Note: In deep learning, a 2-D Array is referred to as a matrix, and arrays with more than dimensions are called tensors.
Attributes
The main attributes of tensors are as follows:
-
Rank: The number of dimensions present within the tensor. For example, a -dimensional tensor has a
rankof , while a -dimensional tensor has arankof . -
Axis: A specific dimension of a tensor. For example, a -dimensional tensor has
axes. -
Shape: The length of each
axisof a tensor. It is possible to reshape a tensor without disturbing any of the elements it contains.
Let’s take a look at each of these attributes with the help of an example. First, consider the tensor shown below:
t = [[1,2,3], [4,5,6], [7,8,9]]
The tensor t has dimensions, so it has a rank of and possible axes.
The elements along the first axis ([1,2,3], [4,5,6], and [7,8,9]) represent arrays, whereas each value within these arrays (the second axis) represents the data.
Since the tensor t has rows and columns, the shape will be [3, 3].
Applications
Tensors provide great flexibility in their dimensions and shape, and therefore they are commonly used to store datasets that contain multi-dimensional information.
Some common applications are as follows:
-
3-D Tensors: Used to store time-series data, e.g., medical scans. Each dimension in the tensor may correspond to frequency, time, and channels of signals, respectively. If data for multiple patients is needed, a 4-D tensor may be used, where the extra dimension represents the sample size.
-
4-D Tensors: Used to store JPEG images. Each dimension may correspond to the sample size, height, width, and color depth of the image, respectively.
-
5-D Tensors: Used to store video data. Each dimension may correspond to the sample size, frames, height, width, and color depth, respectively.
Example
The example below shows how tensors are used in the Pytorch library:
import torch# initialize tensora = torch.tensor([[1,2,3], [4,5,6], [7,8,9]])b = torch.tensor([[1,2,3], [4,5,6], [7,8,9]], dtype=torch.int32)# printing tensorsprint("Tensor A: ", a)print("Tensor B: ", b)# extracting valuesc = a[1][2]d = a[2][1]print("The extracted values are", c, "and", d)e = c.item()f = d.item()print("The extracted values are", e, "and", f)
Explanation
First, two tensor objects, a and b, are initialized using python lists in lines 4 and 5. For b, the tensor is initialized with a specific data type since the dtype attribute is provided during initialization.
You can extract a particular dimension of the tensor through python indexing. For example, in line 12, the tensor a is indexed for the value in the third column of its second row, i.e., a[1][2]. This indexing returns a tensor object, which can be converted into a simple numeric data type using the item function, as shown in lines 16 and 17.
Note: A comprehensive guide on tensors and their operations in
Pytorchcan be found in the Official Guide.
Free Resources
