1

I am building a proof-of-concept around running sub-graphs without recomputing, using tensorflow's partial_run() methods.

Currently I have a simple little python script (see below) that should multiply together two placeholder values and add 1, run as a partial graph. This operation works once, then subsequently fails with the error:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Must run 'setup' before performing partial runs!

Any help as to why this error occurs when the setup has been called would be appreciated.

I am using Ubuntu 16.10 and tensorflow 1.2.1.

code:

import tensorflow as tf

a = tf.placeholder(tf.float32, name='a')
b = tf.placeholder(tf.float32, name='b')

c = tf.multiply(a, b, name='c')

y = tf.add(c, 1, name='y')

ilist = [{a: 1, b: 1}, {a: 2, b: 2}, {a: 1}, {b: 1}, {b: 3}]

with tf.Session() as sess:
    hdle = sess.partial_run_setup([y], [a, b])

    for i, fd in enumerate(ilist):
        y_r = sess.partial_run(hdle, y, feed_dict=fd)

        eout = fd[a] * fd[b] + 1
        print("got {}, expected {}".format(y_r, eout))

full output:

got 2.0, expected 2
Traceback (most recent call last):
  File "merged.py", line 15, in <module>
    y_r = sess.partial_run(hdle, y, feed_dict=fd)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 844, in partial_run
    return self._run(handle, fetches, feed_dict, None, None)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1135, in _do_run
    fetch_list)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Must run 'setup' before performing partial runs!

2 Answers 2

1

This example from API documentation works :

import tensorflow as tf
a = tf.placeholder(tf.float32, shape=[])
b = tf.placeholder(tf.float32, shape=[])
c = tf.placeholder(tf.float32, shape=[])
r1 = tf.add(a, b)
r2 = tf.multiply(r1, c)

with tf.Session() as sess:
    h = sess.partial_run_setup([r1, r2], [a, b, c])
    res = sess.partial_run(h, r1, feed_dict={a: 1, b: 2})
    res = sess.partial_run(h, r2, feed_dict={c: 2})        
    print(res) #prints 6.0

But if we add on more invocation it doesn't . If this doesn't work what is the points in using partial_run.

import tensorflow as tf
a = tf.placeholder(tf.float32, shape=[])
b = tf.placeholder(tf.float32, shape=[])
c = tf.placeholder(tf.float32, shape=[])
r1 = tf.add(a, b)
r2 = tf.multiply(r1, c)

with tf.Session() as sess:
    h = sess.partial_run_setup([r1, r2], [a, b, c])
    res = sess.partial_run(h, r1, feed_dict={a: 1, b: 2})
    res = sess.partial_run(h, r2, feed_dict={c: 2})
    res = sess.partial_run(h, r2, feed_dict={c: 3})

    print(res)
InvalidArgumentError: Must run 'setup' before performing partial runs!
0

The error message is quite explicit: You must run the setup before (every) partial run. I think this is the reference.
Additionally your ilist list of dicts only works for 2 runs. In the third run you only feed a value for a - this won't work. Here is a sample loop that works for me:

ilist = [{a: 1, b: 1}, {a: 2, b: 2}, {a: 1, b: 1}, {a: 1, b: 3}]

with tf.Session() as sess:
    for i, fd in enumerate(ilist):
        hdle = sess.partial_run_setup([y], [a, b])
        y_r = sess.partial_run(hdle, y, feed_dict=fd)

        eout = fd[a] * fd[b] + 1
        print("got {}, expected {}".format(y_r, eout))
6
  • Thanks for your reply! I was under the impression the point of the partial run was so i can avoid feeding placeholders that had already been fed, i.e. the nodes dependant on that placeholder 'cached' their results?
    – Daniel
    Jul 21, 2017 at 9:21
  • Also the documentation shows a single setup and multiple partial runs?
    – Daniel
    Jul 21, 2017 at 9:28
  • @Daniel The documentation shows a partial run on add and on multiply, you can do this too, but atm you're only doing a partial run on y (the add). Is this the goal?
    – dv3
    Jul 21, 2017 at 11:26
  • Currently just toying with ways to deal with two subgraphs feeding an 'upper layer' subgraph, while one of the subgraphs is only rarely fed data. Given how we're looking to scale this I want to avoid using python buffering or running the rarely updated subgraph every time. Came across partial_run() and hoped it would do this. If there's another/better way then I'm happy to hear about it.
    – Daniel
    Jul 21, 2017 at 13:13
  • @Daniel So if it's only the buffering I would look into QueueRunners of Tensorflow - that way you can skip the feeddict and have a higher GPU usage, due to the data feeding not being the bottleneck. I see the problem, but I would separate it into smaller, more dissectible questions.
    – dv3
    Jul 22, 2017 at 8:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.