Visualizing Word Embeddings with TensorBoard

When we wanted to visualize word embeddings in the “Word2vec: Learning Word Embeddings” chapter, we manually implemented the visualization with the t-SNE algorithm. However, we also could use TensorBoard to visualize word embeddings. TensorBoard is a visualization tool provided with TensorFlow. We can use TensorBoard to visualize the TensorFlow variables in our program. This allows us to see how different variables behave over time (for example, model loss/accuracy) so we can identify potential issues in our model.

TensorBoard enables us to visualize scalar values (e.g., loss values over training iterations) and vectors (e.g., model’s layer node activations) as histograms. Apart from this, TensorBoard also allows us to visualize word embeddings. Therefore, it takes all the required code implementation away from us if we need to analyze what the embeddings look like. Next, we’ll see how we can use TensorBoard to visualize word embeddings.

Starting TensorBoard

First, we’ll list the steps for starting TensorBoard. TensorBoard acts ...