This course teaches you how to create, manipulate, and understand tensors for machine learning using PyTorch.

Manipulate Tensor in PyTorch (Mini Course)_468 x 60.png

pytorch.tar.gz

pytorch

Tensors form the fundamental building block of the PyTorch ecosystem. Various components, such as network layers, loss function, gradient descent, and optimizer rely on underlying tensor operations.

In this course, you will get a comprehensive overview of tensors starting with how to create one. You will learn the many different types of tensors as well as how to concatenate and manipulate them by selecting elements, changing the shape, and more.

By the end of this course, you will have a basic understanding of how to work with PyTorch tensors and what they allow you to do.

Basic Pytorch Tensor Manipulation for Machine Learning

# Why tensor type is important

Similar to other frameworks, **PyTorch** too defines its types for its tensor. Some frequently used types are listed in the table below. 

 
>**Notice**: For `integer`, there are also 16-bit, 32-bit, and 64-bit types, signed and unsigned. For `floating`, there is also a 16-bit type.

The tensor type is vital. There are two main reasons.

- It affects speed and memory usage because the GPU has video memory limitations. More bits type takes up more memory and requires more computing resources. For example, a matrix `A` with size 1000\*1000. If the `dtype` is `torch.float32`, this matrix would consume about 3.81MB GPU memory (1000\*1000\*4bytes, each `float32` uses 4 bytes.). If the `dtype` is `torch.double`, this matrix would consume about 7.62MB GPU memory (1000\*1000\*8bytes, each `double` uses 8 bytes.).
- Some APIs have strong requirements for types. For example, when you train a model about a classification task, you need to calculate some metrics. The tensor type of input of some API requires the `torch.long` type. You could check this [example](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html).

In this lesson, we discuss the different types supported by the tensors in PyTorch and show how to cast them into other types.

Data Type	dtype	CPU tensor	GPU tensor
32-bit floating point	torch.float32/torch.float	torch.FloatTensor	torch.cuda.FloatTensor
64-bit floating point	torch.float64/torch.double	torch.DoubleTensor	torch.cuda.DoubleTensor
8-bit integer (signed)	torch.int16	torch.ShortTensor	torch.cuda.ShortTensor
boolean	torch.bool	torch.BoolTensor	torch.cuda.BoolTensor

__default

Tensor Types

Why tensor type is important