Tensors: An Example-based Introduction to the heart of TensorFlow

This post is associated with TensorFlow tensors. Tensor are multi-dimensional arrays that are used in TensorFlow.

Here, I am going to show to define and manipulate tensors by example.

Defining Tensors

Let’s define different kinds of tensors and show their rank using tf.rank function. We should have the following definition:

tensor = tf.constant(0)
print("Print constant tensor {} of rank {}".format(tensor, tf.rank(tensor)))
print("Show full tensor:", tensor)

The output:

Print constant tensor 0 of rank 0
Show full tensor: tf.Tensor(0, shape=(), dtype=int32)

Now, let’s have a tensor with a higher rank.

# NOTE: We use .numpy() to transform tf.tensor to numpy
tensor = tf.constant([1,2,3])
print("Tensor:", tensor)
print("Rank:", tf.rank(tensor).numpy())

NOTE: We use .numpy() to transform tf.tensor to numpy. The output should be as follows:

Tensor: tf.Tensor([1 2 3], shape=(3,), dtype=int32)
Rank: 1

Now, let’s have some basic operations:

x = tf.constant([[1, 1],
                 [1, 1]])
y = tf.constant([[2, 4],
                 [6, 8])
# Add two tensors
print(tf.add(x, y), "\n")
# Add two tensors
print(tf.matmul(x, y), "\n")

Tensor Information

This part is not much different compared to what we learned so far. However, it would be nice to try extracting as much information as possible from a multi-dimensional tensor.

Let’s use tf.ones for our purpose here. It creates an all-one tensor. We set the shape of the tensor and the desired data type.

tensor = tf.ones(shape = [2, 3, 6], dtype = tf.float32)
print('Tensor:', tensor)

Output:

Tensor: tf.Tensor(
[[[1. 1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1. 1.]]
 [[1. 1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1. 1.]
  [1. 1. 1. 1. 1. 1.]]], shape=(2, 3, 6), dtype=float32)

And, here’s how we extract relevant tensor information:

print("Tensor Rank: ", tf.rank(tensor).numpy())
print("Shape: ", tensor.shape)
print("Elements' type", tensor.dtype)
print("The size of the second axis:", tensor.shape[1])
print("The size of the last axis:", tensor.shape[-1])
print("Total number of elements: ", tf.size(tensor).numpy())
print("How many dimensions? ", tensor.ndim)

The output should be as follows:

Tensor Rank:  3
Shape:  (2, 3, 6)
Elements' type <dtype: 'float32'>
The size of the second axis: 3
The size of the last axis: 6
Total number of elements:  36
How many dimensions?  3

Indexing

TensorFlow indexing is aligned with Python indexing. See the following examples.

x = tf.constant([[1, 2, 3],
                 [4, 5, 6],
                 [7, 8, 9]])
# All elements
print(x[:].numpy())
# All elements of the first row
print(x[0,:].numpy())
# First row and last column
print(x[0,-1].numpy())
# From the second row to last and from the third column to last
print(x[1:,2:].numpy)

Data Types

You can change the data type of the TensorFlow tensors for your purpose. This will be done easily by tf.cast.

original_tensor = tf.constant([1, 2, 3, 4], dtype=tf.int32)
print('Original tensor: ', original_tensor)
print("Tensor type before casting: ", original_tensor.dtype)
# Casting to change dtype
casted_tensor = tf.cast(original_tensor, dtype=tf.float32)
print('New tensor: ', casted_tensor)
print("Tensor type after casting: ", casted_tensor.dtype)

Output:

Original tensor:  tf.Tensor([1 2 3 4], shape=(4,), dtype=int32)
Tensor type before casting:  <dtype: 'int32'>
New tensor:  tf.Tensor([1. 2. 3. 4.], shape=(4,), dtype=float32)
Tensor type after casting:  <dtype: 'float32'>

Full source code

Below, you can find the full source code to run:

# Import necessary libraries
import tensorflow as tf
import numpy as np
tensor = tf.constant(0)
print("Print constant tensor {} of rank {}".format(tensor, tf.rank(tensor)))
print("Show full tensor:", tensor)
# NOTE: We use .numpy() to transform tf.tensor to numpy
tensor = tf.constant([1,2,3])
print("Tensor:", tensor)
print("Rank:", tf.rank(tensor).numpy())x = tf.constant([[1, 1],
                 [1, 1]])
y = tf.constant([[2, 4],
                 [6, 8])
# Add two tensors
print(tf.add(x, y), "\n")
# Add two tensors
print(tf.matmul(x, y), "\n")
# We set the shape of the tensor and the desired data type.
tensor = tf.ones(shape = [2, 3, 6], dtype = tf.float32)
print('Tensor:', tensor)
print("Tensor Rank: ", tf.rank(tensor).numpy())
print("Shape: ", tensor.shape)
print("Elements' type", tensor.dtype)
print("The size of the second axis:", tensor.shape[1])
print("The size of the last axis:", tensor.shape[-1])
print("Total number of elements: ", tf.size(tensor).numpy())
print("How many dimensions? ", tensor.ndim)
x = tf.constant([[1, 2, 3],
                 [4, 5, 6],
                 [7, 8, 9]])
# All elements
print(x[:].numpy())
# All elements of the first row
print(x[0,:].numpy())
# First row and last column
print(x[0,-1].numpy())
# From the second row to last and from the third column to last
print(x[1:,2:].numpy)
original_tensor = tf.constant([1, 2, 3, 4], dtype=tf.int32)
print('Original tensor: ', original_tensor)
print("Tensor type before casting: ", original_tensor.dtype)
# Casting to change dtype
casted_tensor = tf.cast(original_tensor, dtype=tf.float32)
print('New tensor: ', casted_tensor)
print("Tensor type after casting: ", casted_tensor.dtype)

Summary

Here, we investigated tensors in TensorFlow with some examples. Tensors are the elements that are used to store and represent arrays in TensorFlow. The main important take away from this tutorial is understanding what tensors are and how you can define and manipulate them. Try to explore more by running the above code to see if you can confirm and validate it. I hope you find this tutorial useful.