TensorFlow Variable Class and load() Function

Neural Network Tutorials - Herong's Tutorial Examples

∟TensorFlow Variable Class and load() Function

This section provides a tutorial example on how to create a TensoFlow variable object that acts like a tensor operation and provides extra functions to manage its output tensor value. The v.load(a, ss) function is the most efficient way assign new values to 'Variable' tensors.

Before building any more complex tensor flow graphs, we need to take a closer look at "Variable" tensor operations so we can use them more efficiently.

A "Variable" tensor operation is actually an object of the tf.Variable class, which contains a tensor operation and acts like a tensor operation. To create a Variable object, we call the tf.Variable class constructor as shown below:

v = tf.Variable(a)

The Variable object, v, created above contains a tensor operation and other properties/functions. The given input tensor is kept as the initial tensor, which provides the shape, data type and initial value to the output tensor. But the initial value is not populated to the output tensor yet at the time of creation.

There are 3 ways to populate value to a "Variable" tensor:

Using an "initialize" operation to assign the its initial tensor to its output tensor,
Using an "assign" operation to assign a different tensor.
Using the load() function to load a different tensor under a session context.

Here are some extra functions that are supported on "Variable" tensor operations:

v.assign(a) - Creates a tensor operation whose job is to assign the given input tensor, a, to the variable, v, as its output tensor.

v.load(a, s) - Loads the input tensor, a, to to the variable, v, as its output tensor under the session, s.

v.eval(s) - Returns the value of the output tensor from variable, v, under the session, s.

Here is an example script on how to Tensor Variable objects, update_variable_in_loop.py:

#- update_variable_in_loop.py
#- Copyright (c) 2019 HerongYang.com. All Rights Reserved.
#
import tensorflow as tf

# create 3 variable objects to hold float scalars,
ave = tf.Variable(0.0)
min = tf.Variable(0.0)
max = tf.Variable(0.0)

# using "initialize" operation on "ave"
ave_init = tf.variables_initializer([ave])

# using "assign" operation on "min"
min_assign = min.assign(999999.0)

# create a TensorFlow session object to run the graph
ss = tf.Session()

# populate initial value to "ave"
out = ss.run(ave_init)

# assign value to "min"
out = ss.run(min_assign)

# load value to "max"
max.load(-999999.0, ss)

def print_stats(ss, ave, min, max):
  print("   Average = "+str(ave.eval(ss)))
  print("   Minimum = "+str(min.eval(ss)))
  print("   Maximum = "+str(max.eval(ss)))

print("Stats before loop:")
print_stats(ss, ave, min, max)

import random
for i in range(10):
  x = random.random()

  if x > max.eval(ss):
    max.load(x, ss)

  if x < min.eval(ss):
    min.load(x, ss)

  ave_upd = ave.assign(ave + x)
  out = ss.run(ave_upd)

ave.load(ave.eval(ss)/10, ss)

print("Stats after loop:")
print_stats(ss, ave, min, max)

If you run it, you will get something like:

herong$ python3 update_variable_in_loop.py

Stats before loop:
   Average = 0.0
   Minimum = 999999.0
   Maximum = -999999.0

Stats after loop:
   Average = 0.3562246
   Minimum = 0.061167717
   Maximum = 0.7353279