Defining Variables and Output in TensorFlow

Defining variables in TensorFlow

Variables play an important role in TensorFlow. A variable is essentially a tensor with a specific shape defining how many dimensions the variable will have and the size of each dimension. However, unlike a regular TensorFlow tensor, variables are mutable, meaning that the value of the variables can change after they are defined. This is an ideal property to have to implement the parameters of a learning model (for example, neural network weights), where the weights change slightly after each step of learning. For example, if we define a variable with x = tf.Variable(0,dtype=tf.int32), we can change the value of that variable using a TensorFlow operation such as tf.assign(x,x+1). However, if we define a tensor such as x = tf.constant(0,dtype=tf.int32), we can’t change the value of the tensor as we could for a variable. It should stay 0 until the end of the program execution.

Variable creation is quite simple. In our sigmoid example, we already created two variables, W and b. When creating a variable, a few things are extremely important. We’ll list them here and discuss each in detail in the following paragraphs:

• Variable shape
• Initial values
• Data type
• Name (optional)

Variable shape

The variable shape is a list of the [x,y,z,...] format. Each value in the list indicates how large the corresponding dimension or axis is. For instance, if we require a 2D tensor with 50 rows and 10 columns as the variable, the shape would be equal to [50,10].

The dimensionality of the variable (that is, the length of the shape vector) is recognized as the rank of the tensor in TensorFlow. Do not confuse this with the rank of a matrix.

Note: Tensor rank in TensorFlow indicates the dimensionality of the tensor; for a two-dimensional matrix, rank = 2.

Initial values

Next, a variable requires an initial value to be initialized with. TensorFlow provides several different initializers for our convenience, including constant initializers and normal distribution initializers. Here are a few popular TensorFlow initializers we can use to initialize variables:

• tf.initializers.Zeros
• tf.initializers.Constant
• tf.initializers.RandomNormal
• tf.initializers.GlorotUniform

The shape of the variable can be provided as a part of the initializer as follows:

tf.initializers.RandomUniform(minval = -0.1, maxval = 0.1)(shape = [10,5])