I'm implementing an encoder-decoder rnn by using tf.contrib.cudnn_rnn.CudnnGRU()
as the encoder and I've found a problem:
I want to reuse the variables so I can create the same model but use it with other data and to put it simple this would be the code to reproduce my problem:
tf.reset_default_graph()
def create_model():
return tf.contrib.cudnn_rnn.CudnnGRU(num_layers=1, num_units=100,
direction='unidirectional')
# (time, batch_size, num_inputs)
x = tf.random_normal((100, 16, 100))
with tf.variable_scope('model') as scope:
model_1 = create_model()
rnn_out_1, rnn_state_1 = model_1(x)
scope.reuse_variables()
model_2 = create_model()
rnn_out_2, rnn_state_2 = model_2(x)
This throws the following error:
Variable model/cudnn_gru_1/opaque_kernel does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?
So the second model is trying to find the model/cudnn_gru_1/opaque_kernel
variable but can not find it because it should be looking for model/cudnn_gru/opaque_kernel:0
.
The thing is I don't know why this is happening since by following the Variables tensorflow reference it seems to be ok. On the other hand, I also tried to write it differently as tensorflow doc states that my above implementation and the one I'm showing next do actually the same:
tf.reset_default_graph()
def create_model():
return tf.contrib.cudnn_rnn.CudnnGRU(num_layers=1, num_units=100,
direction='unidirectional')
# (time, batch_size, num_inputs)
x = tf.random_normal((100, 16, 100))
with tf.variable_scope('model'):
model_1 = create_model()
rnn_out_1, rnn_state_1 = model_1(x)
with tf.variable_scope('model', reuse=True):
model_2 = create_model()
rnn_out_2, rnn_state_2 = model_2(x)
This second way is actually working (or at least I think it is). So I don't really know what I'm doing wrong in the first implementation, I'm also not sure on wether both implementations should be doing the same (which I think they should). So does anyone please can help me figure out what I'm doing wrong or the things that I'm not able to understand properly?
Thanks in advance