I would like to be able to log to tensorboard some intermediate results during loss calculation. This seems like a pretty basic thing to me, since for instance in some detection CNNs the loss is made of a confidence component and a localisation one, and monitoring each of them can be important.
For what I've searched, I think this should be accomplished by using tf.summary
. In this other question that's what they propose, however it doesn't work for me.
Below is a toy example, where I try to use tf.summary
. The point is to monitor tf.reduce_mean(sign)
(just for the sake of being able to log something).
import tensorflow as tf
file_writer = tf.summary.create_file_writer('./logs')
file_writer.set_as_default()
def huber_loss(y_true, y_pred):
diff = tf.subtract(y_true, y_pred)
sign = tf.cast(tf.less(tf.abs(diff), 1.0), tf.float32)
tf.summary.scalar('mean sign', tf.reduce_mean(sign))
loss = 0.5 * diff * diff * sign + (tf.abs(diff) - 0.5) * (1.0 - sign)
return tf.reduce_sum(loss)
x = tf.random.uniform(minval=0, maxval=1, shape=(1000, 4), dtype=tf.float32)
y = tf.multiply(tf.reduce_sum(x, axis=-1), 5)
model = tf.keras.Sequential([
tf.keras.layers.Dense(16, input_shape=[4], activation='relu'),
tf.keras.layers.Dense(32, activation='relu'),
tf.keras.layers.Dense(1)
])
model.compile(loss=huber_loss, optimizer='adam')
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir='./logs', update_freq='batch')
history = model.fit(x, y, epochs=10, callbacks=[tensorboard_callback])
When I run that code, the plot created in tensorboard for mean sign
has only one value at step 0: Plotted loss has only one value at step 0. (The plot for the total loss is fine however, but that's not what I'm interested on).
So can anybody tell me what's the problem in the code above, or how can that be achieved?