TensorFlow 2 Architecture: Building and Executing Graphs

Build graphs automatically

Let’s now go over what TensorFlow does when we execute TensorFlow operations.

When we call a function decorated by tf.function(), such as the layer() function, there is quite a bit happening in the background. First, TensorFlow will trace all the TensorFlow operations taking place in the function and build the computational graph automatically.

In fact, tf.function() will return a function that executes the built dataflow graph when invoked. Therefore, tf.function() is a multistage process, where it first builds the dataflow graph and then executes it. Additionally, since TensorFlow traces each line in the function, if something goes wrong, TensorFlow can point to the exact line that is causing the issue.

Computational graph of sigmoid example

In our sigmoid example, the computational, or dataflow, graph would look like the figure below. A single element or vertex of the graph is called a node. There are two main types of objects in this graph: operations and tensors. In the preceding example, tf.nn.sigmoid is an operation and h is a tensor: