12

I am trying to make 2 conv layers share the same weights, however, it seems the API does not work.

import tensorflow as tf

x = tf.random_normal(shape=[10, 32, 32, 3])

with tf.variable_scope('foo') as scope:
    conv1 = tf.contrib.layers.conv2d(x, 3, [2, 2], padding='SAME', reuse=True, scope=scope)
    print(conv1.name)

    conv2 = tf.contrib.layers.conv2d(x, 3, [2, 2], padding='SAME', reuse=True, scope=scope)
    print(conv2.name)

It prints out

foo/foo/Relu:0
foo/foo_1/Relu:0

Changing from tf.contrib.layers.conv2d to tf.layers.conv2d does not solve the problem.

It has the same problem with tf.layers.conv2d:

import tensorflow as tf

x = tf.random_normal(shape=[10, 32, 32, 3])

conv1 = tf.layers.conv2d(x, 3, [2, 2], padding='SAME', reuse=None, name='conv')
print(conv1.name)
conv2 = tf.layers.conv2d(x, 3, [2, 2], padding='SAME', reuse=True, name='conv')
print(conv2.name)

gives

conv/BiasAdd:0
conv_2/BiasAdd:0
5
  • for the first example why is reuse true even on the first conv?
    – Steven
    Commented Mar 17, 2017 at 18:47
  • @Steven I've tried all combinations to put reuse=True, but none of them to be effective. Could you help to give a short example to how make the weights to be sharing by using tf.layers.conv2d ?
    – Xingdong
    Commented Mar 17, 2017 at 19:06
  • I usually do it by hand i.e. create the weights and then pass them in. Then I have the weight variable that I can reuse by simply using the same variable. I can show an example of that?
    – Steven
    Commented Mar 17, 2017 at 20:03
  • In the first conv2d layer conv1 = tf.layers.conv2d(x, 3, [2, 2], padding='SAME', reuse=None), Should the reuse argument be None?
    – William
    Commented Nov 6, 2017 at 16:44
  • conv1 and conv2 are tensors which are different, but their weights (trainable variables) are the same (shared)
    – thinkdeep
    Commented Sep 29, 2018 at 3:28

1 Answer 1

17

In the code you wrote, variables do get reused between the two convolution layers. Try this :

import tensorflow as tf

x = tf.random_normal(shape=[10, 32, 32, 3])

conv1 = tf.layers.conv2d(x, 3, [2, 2], padding='SAME', reuse=None, name='conv')

conv2 = tf.layers.conv2d(x, 3, [2, 2], padding='SAME', reuse=True, name='conv')

print([x.name for x in tf.global_variables()])

# prints
# [u'conv/kernel:0', u'conv/bias:0']

Note that only one weight and one bias tensor has been created. Even though they share the weights, the layers do not share the actual computation. Hence you see the two different names for the operations.

1
  • 5
    Note that you can set reuse=tf.AUTO_REUSE so that you don't have to set it to False/None for the first call then True for subsequent ones. This keeps you from needing a special case for the first call.
    – Nathan
    Commented Nov 28, 2017 at 22:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.