In this tutorial, we describe how to define and initialize variables in TensorFlow. It is a crucial step as without having parameters, training, updating, saving, restoring and any other operations cannot be performed.
Introduction
variablesdefineinitialize
Creating variables
For a variable generation, the class of tf.Variable() will be used. When we define a variable, we basically pass a tensor and its value to the graph. Basically, the following will happen:
- A
variabletensor that holds a value will be pass to the graph. - By using tf.assign, an initializer set initial variable value.
Some arbitrary variables can be defined as follows:
import tensorflow as tf
import xlrd
import matplotlib.pyplot as plt
import os
from sklearn.utils import check_random_state
# Generating artificial data.
n = 50
XX = np.arange(n)
rs = check_random_state(0)
YY = rs.randint(-20, 20, size=(n,)) + 2.0 * XX
data = np.stack([XX,YY], axis=1)
#######################
## Defining flags #####
#######################
tf.app.flags.DEFINE_integer('num_epochs', 50, 'The number of epochs for training the model. Default=50')
# Store all elemnts in FLAG structure!
FLAGS = tf.app.flags.FLAGSIn the above script, line 15 gets the list of all defined variables from the defined graph. The “name” key, define a specific name for each variable on the graph
Initialization
Initializers of the variables must be run before all other operations in the model. For an analogy, we can consider the starter of the car. Instead of running an initializer, variables can be restored too from saved models such as a checkpoint file. Variables can be initialized globally, specifically, or from other variables. We investigate different choices in the subsequent sections.
Initializing Specific Variables
By using tf.variables_initializer, we can explicitly command the TensorFlow to only initialize
# "variable_list_custom" is the list of variables that we want to initialize. variable_list_custom = [weights, custom_variable] # The initializer init_custom_op = tf.variables_initializer(var_list=all_variables_list)
Noted that custom initialization does not mean that we don’t need to initialize other variables! All variables that some operations will be done upon them over the graph, must be initialized or restored from saved variables. This only let us realize how we can initialize specific variables by hand.
Golobal variable initialization
All variables can be initialized at once using the tf.global_variables_initializer(). This op must be run after the model being fully constructed. The script is as below:
# Add an op to initialize the variables. init_all_op = tf.global_variables_initializer() # Method-2 init_all_op = tf.variables_initializer(var_list=all_variables_list)
Both the above methods are identical. We only provide the second one to demonstrate that the tf.global_variables_initializer() is nothing but tf.variables_initializer when you yield all the variables as its input argument.
Initialization of a variable using other existing variables
New variables can be initialized using other existing variables’ initial values by taking the values using initialized_value().
# Create another variable with the same value as 'weights'. WeightsNew = tf.Variable(weights.initialized_value(), name="WeightsNew") # Now, the variable must be initialized. init_WeightsNew_op = tf.variables_initializer(var_list=[WeightsNew])
As it can be seen from the above script, the WeightsNew variable is initialized with the values of the weights predefined value.
[thrive_leads id=’1438′]
Running the session
All we did so far was to define the
with tf.Session() as sess:
# Run the initializer operation.
sess.run(init_all_op)
sess.run(init_custom_op)
sess.run(init_WeightsNew_op)Each of the initializers has been run separated using a session.
Summary
In this tutorial, we walked through the variable creation and initialization. The global, custom and inherited variable initialization have been investigated. In future posts, we investigate how to save and restore the variables. Restoring a variable eliminate the necessity of its initialization.