I am attempting to use a batch norm layer from TensorFlow-Slim as follows:
net = ...
net = slim.batch_norm(net, scale = True, is_training = self.isTraining,
updates_collections = None, decay = 0.9)
net = tf.nn.relu(net)
net = ...
I train with:
self.optimizer = slim.learning.create_train_op(self.model.loss,
tf.train.MomentumOptimizer(learning_rate = self.learningRate,
momentum = 0.9, use_nesterov = True)
optimizer = self.sess.run([self.optimizer],
feed_dict={self.model.isTraining:True})
I load the saved weights with:
net = model.Model(sess,width,height,channels,weightDecay)
savedWeightsDir = './savedWeights/'
saver = tf.train.Saver(max_to_keep = 5)
checkpointStr = tf.train.latest_checkpoint(savedWeightsDir)
sess.run(tf.global_variables_initializer())
saver.restore(sess, checkpointStr)
global_step = tf.contrib.framework.get_or_create_global_step()
and I infer with:
inf = self.sess.run([self.softmax],
feed_dict = {self.imageBatch:imageBatch,self.isTraining:False})
Of course I left out a lot and paraphrased some code, but I think this is all that the batch norm touches. The odd thing is, if I set isTraining:True, I get much better results. Could it be something with loading in the weights - perhaps the batch norm values are not saved? Is there anything obviously wrong in the code? Thank you.