315

I have been using the introductory example of matrix multiplication in TensorFlow.

matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)

When I print the product, it is displaying it as a Tensor object:

<tensorflow.python.framework.ops.Tensor object at 0x10470fcd0>

But how do I know the value of product?

The following doesn't help:

print product
Tensor("MatMul:0", shape=TensorShape([Dimension(1), Dimension(1)]), dtype=float32)

I know that graphs run on Sessions, but isn't there any way I can check the output of a Tensor object without running the graph in a session?

0

24 Answers 24

282

The easiest[A] way to evaluate the actual value of a Tensor object is to pass it to the Session.run() method, or call Tensor.eval() when you have a default session (i.e. in a with tf.Session(): block, or see below). In general[B], you cannot print the value of a tensor without running some code in a session.

If you are experimenting with the programming model, and want an easy way to evaluate tensors, the tf.InteractiveSession lets you open a session at the start of your program, and then use that session for all Tensor.eval() (and Operation.run()) calls. This can be easier in an interactive setting, such as the shell or an IPython notebook, when it's tedious to pass around a Session object everywhere. For example, the following works in a Jupyter notebook:

with tf.Session() as sess:  print(product.eval()) 

This might seem silly for such a small expression, but one of the key ideas in Tensorflow 1.x is deferred execution: it's very cheap to build a large and complex expression, and when you want to evaluate it, the back-end (to which you connect with a Session) is able to schedule its execution more efficiently (e.g. executing independent parts in parallel and using GPUs).


[A]: To print the value of a tensor without returning it to your Python program, you can use the tf.print() operator, as Andrzej suggests in another answer. According to the official documentation:

To make sure the operator runs, users need to pass the produced op to tf.compat.v1.Session's run method, or to use the op as a control dependency for executed ops by specifying with tf.compat.v1.control_dependencies([print_op]), which is printed to standard output.

Also note that:

In Jupyter notebooks and colabs, tf.print prints to the notebook cell outputs. It will not write to the notebook kernel's console logs.

[B]: You might be able to use the tf.get_static_value() function to get the constant value of the given tensor if its value is efficiently calculable.

6
  • 23
    It is possible to get some attributes of a Tensor without calling Session.run(). For example, you can call tensor.get_shape(). In many cases, this gives enough information to debug. Commented Apr 23, 2016 at 18:18
  • 8
    See also And's answer about the tf.Print op below. I keep finding this stackoverflow answer while googling for "tensorflow print" and this top answer makes it sound like there is no tf.Print op. Commented Apr 23, 2016 at 18:19
  • 2
    I added some caveats to the answer, so it should be clearer now. (I don't think the original questioner was interested in getting the shape of a tensor, just the value.)
    – mrry
    Commented Apr 23, 2016 at 22:52
  • 1
    Is there a way to save to a file instead of print to console (via tf.Print)?
    – thang
    Commented Aug 18, 2016 at 18:26
  • 9
    tf.Session() doesn't work in Tensorflow 2. You can use tf.compat.v1.Session() instead.
    – mic
    Commented May 27, 2020 at 5:04
168

While other answers are correct that you cannot print the value until you evaluate the graph, they do not talk about one easy way of actually printing a value inside the graph, once you evaluate it.

The easiest way to see a value of a tensor whenever the graph is evaluated (using run or eval) is to use the Print operation as in this example:

# Initialize session
import tensorflow as tf
sess = tf.InteractiveSession()

# Some tensor we want to print the value of
a = tf.constant([1.0, 3.0])

# Add print operation
a = tf.Print(a, [a], message="This is a: ")

# Add more elements of the graph using a
b = tf.add(a, a)

Now, whenever we evaluate the whole graph, e.g. using b.eval(), we get:

I tensorflow/core/kernels/logging_ops.cc:79] This is a: [1 3]
6
  • 40
    it is VERY Important that you use the a from a=tf.print into something else! tf.print(a,[a]) won't do anything otherwise Commented Oct 14, 2016 at 21:58
  • 6
    We can just use a.eval() then ! Commented Jun 21, 2017 at 6:56
  • 3
    @FabioDias I don't think I got your point? Could you please kindly elaborate when you have time...
    – yuqli
    Commented Jul 27, 2018 at 17:32
  • 10
    Please note that tf.Print() has been deprecated and (now) removed. Instead use tf.print(). See docs: tensorflow.org/api_docs/python/tf/Print and tensorflow.org/api_docs/python/tf/print.
    – Hephaestus
    Commented Dec 7, 2018 at 8:08
  • 1
    wow I am just surprised to see my own comment one year later @yuqli but now I do understand his point. See this post, which is still about the deprecated API but the ideas are probably similar.
    – yuqli
    Commented Aug 13, 2019 at 3:40
28

Reiterating what others said, its not possible to check the values without running the graph.

A simple snippet for anyone looking for an easy example to print values is as below. The code can be executed without any modification in ipython notebook

import tensorflow as tf

#define a variable to hold normal random values 
normal_rv = tf.Variable( tf.truncated_normal([2,3],stddev = 0.1))

#initialize the variable
init_op = tf.initialize_all_variables()

#run the graph
with tf.Session() as sess:
    sess.run(init_op) #execute init_op
    #print the random values that we sample
    print (sess.run(normal_rv))

Output:

[[-0.16702934  0.07173464 -0.04512421]
 [-0.02265321  0.06509651 -0.01419079]]
1
  • 2
    Just FYI: WARNING:tensorflow:From <ipython-input-25-8583e1c5b3d6>:1: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Use 'tf.global_variables_initializer' instead. Commented Mar 7, 2017 at 2:32
21

In Tensorflow 1.x

import tensorflow as tf
tf.enable_eager_execution()
matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)

#print the product
print(product)         # tf.Tensor([[12.]], shape=(1, 1), dtype=float32)
print(product.numpy()) # [[12.]]

With Tensorflow 2.x, eager mode is enabled by default. so the following code works with TF2.0.

import tensorflow as tf
matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)

#print the product
print(product)         # tf.Tensor([[12.]], shape=(1, 1), dtype=float32)
print(product.numpy()) # [[12.]]
3
  • 1
    I have Installed TensorFlow version 1.13.2 and enabled eager execution (checked if running with tf.executing_eagerly()) and getting the error 'Tensor' object has no attribute 'numpy' when trying to evaluate the tensor value inside the custom loss function. I would really appreciate any help to solve the issue. Commented Jul 22, 2019 at 14:16
  • 1
    @NikoGamulin make sure you have put tf.compat.v1.enable_eager_execution() at the beginning of your script. I have version 1.14.0, I am running my script on PyCharm, and tensor.numpy() works Commented Jul 24, 2019 at 7:06
  • 1
    @NikoGamulin that error shows up only when you are trying to access a tensor in Graph mode. I think, may be eager execution was not enabled properly. In order to check eager execution, just define a a=tf.constant(2.0), b=tf.constant(3.0), print(tf.add(a,b)). If you see answer as 5.0 then eager was enabled properly. Commented Jul 24, 2019 at 21:40
20

No, you can not see the content of the tensor without running the graph (doing session.run()). The only things you can see are:

  • the dimensionality of the tensor (but I assume it is not hard to calculate it for the list of the operations that TF has)
  • type of the operation that will be used to generate the tensor (transpose_1:0, random_uniform:0)
  • type of elements in the tensor (float32)

I have not found this in documentation, but I believe that the values of the variables (and some of the constants are not calculated at the time of assignment).


Take a look at this example:

import tensorflow as tf
from datetime import datetime
dim = 7000

The first example where I just initiate a constant Tensor of random numbers run approximately the same time irrespectibly of dim (0:00:00.003261)

startTime = datetime.now()
m1 = tf.truncated_normal([dim, dim], mean=0.0, stddev=0.02, dtype=tf.float32, seed=1)
print datetime.now() - startTime

In the second case, where the constant is actually gets evaluated and the values are assigned, the time clearly depends on dim (0:00:01.244642)

startTime = datetime.now()
m1 = tf.truncated_normal([dim, dim], mean=0.0, stddev=0.02, dtype=tf.float32, seed=1)
sess = tf.Session()
sess.run(m1)
print datetime.now() - startTime

And you can make it more clear by calculating something (d = tf.matrix_determinant(m1), keeping in mind that the time will run in O(dim^2.8))

P.S. I found were it is explained in documentation:

A Tensor object is a symbolic handle to the result of an operation, but does not actually hold the values of the operation's output.

16

tf.keras.backend.eval is useful for evaluating small expressions.

tf.keras.backend.eval(op)

TF 1.x and TF 2.0 compatible.


Minimal Verifiable Example

from tensorflow.keras.backend import eval

m1 = tf.constant([[3., 3.]])
m2 = tf.constant([[2.],[2.]])

eval(tf.matmul(m1, m2))
# array([[12.]], dtype=float32)

This is useful because you do not have to explicitly create a Session or InteractiveSession.

3
  • 1
    What might be happening here? AttributeError: 'Tensor' object has no attribute '_numpy' Commented Jun 26, 2021 at 19:55
  • This works as of 6/27/2022 for those that may stumble upon it Commented Jun 27, 2022 at 23:23
  • @WestCoastProjects You can turn on eager execution if needed using tf.config.run_functions_eagerly(True) Commented Apr 30, 2023 at 9:49
15

I think you need to get some fundamentals right. With the examples above you have created tensors (multi dimensional array). But for tensor flow to really work you have to initiate a "session" and run your "operation" in the session. Notice the word "session" and "operation". You need to know 4 things to work with tensorflow:

  1. tensors
  2. Operations
  3. Sessions
  4. Graphs

Now from what you wrote out you have given the tensor, and the operation but you have no session running nor a graph. Tensor (edges of the graph) flow through graphs and are manipulated by operations (nodes of the graph). There is default graph but you can initiate yours in a session.

When you say print , you only access the shape of the variable or constant you defined.

So you can see what you are missing :

 with tf.Session() as sess:     
           print(sess.run(product))
           print (product.eval())

Hope it helps!

11

In Tensorflow 2.0+ (or in Eager mode environment) you can call .numpy() method:

import tensorflow as tf

matrix1 = tf.constant([[3., 3.0]])
matrix2 = tf.constant([[2.0],[2.0]])
product = tf.matmul(matrix1, matrix2)

print(product.numpy()) 
1
  • tf.print(product) as well gives me the same output as print(product.numpy()) with TF 2.0.
    – HUSMEN
    Commented Oct 20, 2019 at 20:38
8

Based on the answers above, with your particular code snippet you can print the product like this:

import tensorflow as tf
#Initialize the session
sess = tf.InteractiveSession()

matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)

#print the product
print(product.eval())

#close the session to release resources
sess.close()
8

I am not sure if I am missing here, but I think the easiest and best way to do it is using tf.keras.backend.get_value API.

print(product)
>>tf.Tensor([[12.]], shape=(1, 1), dtype=float32)
print(tf.keras.backend.get_value(product))
>>[[12.]]
7

You can check the output of a TensorObject without running the graph in a session, by enabling eager execution.

Simply add the following two lines of code: import tensorflow.contrib.eager as tfe tfe.enable_eager_execution()

right after you import tensorflow.

The output of print product in your example will now be: tf.Tensor([[ 12.]], shape=(1, 1), dtype=float32)

Note that as of now (November 2017) you'll have to install a Tensorflow nightly build to enable eager execution. Pre-built wheels can be found here.

6

You can use Keras, one-line answer will be to use eval method like so:

import keras.backend as K
print(K.eval(your_tensor))
0
5

Please note that tf.Print() will change the tensor name. If the tensor you seek to print is a placeholder, feeding data to it will fail as the original name will not be found during feeding. For example:

import tensorflow as tf
tens = tf.placeholder(tf.float32,[None,2],name="placeholder")
print(eval("tens"))
tens = tf.Print(tens,[tens, tf.shape(tens)],summarize=10,message="tens:")
print(eval("tens"))
res = tens + tens
sess = tf.Session()
sess.run(tf.global_variables_initializer())

print(sess.run(res))

Output is:

python test.py
Tensor("placeholder:0", shape=(?, 2), dtype=float32)
Tensor("Print:0", shape=(?, 2), dtype=float32)
Traceback (most recent call last):
[...]
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'placeholder' with dtype float
5

You should think of TensorFlow Core programs as consisting of two discrete sections:

  • Building the computational graph.
  • Running the computational graph.

So for the code below you just Build the computational graph.

matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)

You need also To initialize all the variables in a TensorFlow program , you must explicitly call a special operation as follows:

init = tf.global_variables_initializer()

Now you build the graph and initialized all variables ,next step is to evaluate the nodes, you must run the computational graph within a session. A session encapsulates the control and state of the TensorFlow runtime.

The following code creates a Session object and then invokes its run method to run enough of the computational graph to evaluate product :

sess = tf.Session()
// run variables initializer
sess.run(init)

print(sess.run([product]))
3

Try this simple code! (it is self explanatory)

import tensorflow as tf
sess = tf.InteractiveSession() # see the answers above :)
x = [[1.,2.,1.],[1.,1.,1.]]    # a 2D matrix as input to softmax
y = tf.nn.softmax(x)           # this is the softmax function
                               # you can have anything you like here
u = y.eval()
print(u)
3

In Tensorflow V2, Print value of tensor using: tf.keras.backend.print_tensor(x, message='')

1
  • 404 error on the link Commented Jun 23 at 17:50
2

I didn't find it easy to understand what is required even after reading all the answers until I executed this. TensofFlow is new to me too.

def printtest():
x = tf.constant([1.0, 3.0])
x = tf.Print(x,[x],message="Test")
init = (tf.global_variables_initializer(), tf.local_variables_initializer())
b = tf.add(x, x)
with tf.Session() as sess:
    sess.run(init)
    print(sess.run(b))
    sess.close()

But still you may need the value returned by executing the session.

def printtest():
    x = tf.constant([100.0])
    x = tf.Print(x,[x],message="Test")
    init = (tf.global_variables_initializer(), tf.local_variables_initializer())
    b = tf.add(x, x)
    with tf.Session() as sess:
        sess.run(init)
        c = sess.run(b)
        print(c)
        sess.close()
2

You could print out the tensor value in session as follow:

import tensorflow as tf

a = tf.constant([1, 1.5, 2.5], dtype=tf.float32)
b = tf.constant([1, -2, 3], dtype=tf.float32)
c = a * b

with tf.Session() as sess:
    result = c.eval()
   
print(result)
1
  • AttributeError: module 'tensorflow' has no attribute 'Session' Commented Jun 23 at 17:49
1

Basically, in tensorflow when you create a tensor of any sort they are created and stored inside which can be accessed only when you run a tensorflow session. Say you have created a constant tensor
c = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
Without running a session, you can get
- op: An Operation. Operation that computes this tensor.
- value_index: An int. Index of the operation's endpoint that produces this tensor.
- dtype: A DType. Type of elements stored in this tensor.

To get the values you can run a session with the tensor you require as:

with tf.Session() as sess:
    print(sess.run(c))
    sess.close()

The output will be something like this:

array([[1., 2., 3.], [4., 5., 6.]], dtype=float32)

1

Enable the eager execution which is introduced in tensorflow after version 1.10. It's very easy to use.

# Initialize session
import tensorflow as tf
tf.enable_eager_execution()


# Some tensor we want to print the value of
a = tf.constant([1.0, 3.0])

print(a)
1

Using tips provided in https://www.tensorflow.org/api_docs/python/tf/print I use the log_d function to print formatted strings.

import tensorflow as tf

def log_d(fmt, *args):
    op = tf.py_func(func=lambda fmt_, *args_: print(fmt%(*args_,)),
                    inp=[fmt]+[*args], Tout=[])
    return tf.control_dependencies([op])


# actual code starts now...

matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)

with log_d('MAT1: %s, MAT2: %s', matrix1, matrix2): # this will print the log line
    product = tf.matmul(matrix1, matrix2)

with tf.Session() as sess:
    sess.run(product)
0
import tensorflow as tf
sess = tf.InteractiveSession()
x = [[1.,2.,1.],[1.,1.,1.]]    
y = tf.nn.softmax(x)           

matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)

print(product.eval())
tf.reset_default_graph()
sess.close()
0

tf.Print is now deprecated, here's how to use tf.print (lowercase p) instead.

While running a session is a good option, it is not always the way to go. For instance, you may want to print some tensor in a particular session.

The new print method returns a print operation which has no output tensors:

print_op = tf.print(tensor_to_print)

Since it has no outputs, you can't insert it in a graph the same way as you could with tf.Print. Instead, you can you can add it to control dependencies in your session in order to make it print.

sess = tf.compat.v1.Session()
with sess.as_default():
  tensor_to_print = tf.range(10)
  print_op = tf.print(tensor_to_print)
with tf.control_dependencies([print_op]):
  tripled_tensor = tensor_to_print * 3
sess.run(tripled_tensor)

Sometimes, in a larger graph, maybe created partly in subfunctions, it is cumbersome to propagate the print_op to the session call. Then, tf.tuple can be used to couple the print operation with another operation, which will then run with that operation whichever session executes the code. Here's how that is done:

print_op = tf.print(tensor_to_print)
some_tensor_list = tf.tuple([some_tensor], control_inputs=[print_op])
# Use some_tensor_list[0] instead of any_tensor below.
-2

Question: How to print the value of a Tensor object in TensorFlow?

Answer:

import tensorflow as tf

# Variable
x = tf.Variable([[1,2,3]])

# initialize
init = (tf.global_variables_initializer(), tf.local_variables_initializer())

# Create a session
sess = tf.Session()

# run the session
sess.run(init)

# print the value
sess.run(x)

Not the answer you're looking for? Browse other questions tagged or ask your own question.