In a general tensorflow setup like
model = construct_model()
with tf.Session() as sess:
train_model(sess)
Where construct_model()
contains the model definition including random initialization of weights (tf.truncated_normal
) and train_model(sess)
executes the training of the model -
Which seeds do I have to set where to ensure 100% reproducibility between repeated runs of the code snippet above? The documentation for tf.random.set_random_seed
may be concise, but left me a bit confused. I tried:
tf.set_random_seed(1234)
model = construct_model()
with tf.Session() as sess:
train_model(sess)
But got different results each time.
sess = tf.Session(config=tf.ConfigProto(inter_op_parallelism_threads=1,intra_op_parallelism_threads=1)
sess = tf.Session...
in the comments does not turn off the GPU, as observed bywatch nvidia-smi
(in the case of an nvidia gpu, as on AWS EC2 p2.xlarge instances)