3

This paper suggests the following algorithm that incorporates anomalous data points into the training process:

Algorithm 1 Autoencoding Binary Classifiers

while not converged do:
  Sample minibatch {(x1, y1), · · · , (xK , yK )} from dataset 
  Compute the gradient of θ as g0 (using custom loss function that incorporates the label of x) 
  Perform SGD-update for θ with gθ 
end while

I have the following code for an autoencoder implemented using keras:

class AnomalyDetector(Model):
def __init__(self):
    super(AnomalyDetector, self).__init__()
    self.encoder = tf.keras.Sequential([
      layers.Dense(32, activation="relu"),
      layers.Dense(16, activation="relu"),
      layers.Dense(8, activation="relu")])
    
    self.decoder = tf.keras.Sequential([
      layers.Dense(16, activation="relu"),
      layers.Dense(32, activation="relu"),
      layers.Dense(140, activation="sigmoid")])
    
  def call(self, x):
    encoded = self.encoder(x)
    decoded = self.decoder(encoded)
    return decoded

autoencoder = AnomalyDetector()

My main problem is: how do I include the labels in a customly defined loss function if an autoencoder's input and output must be X. Assuming I encode the label as a feature, during inference, the label won't be available, so I am not sure how to implement the algorithm described by the paper.

1 Answer 1

2
+25

Add the labels to y and split them up again in your loss function:

# dummy input
x = tf.random.uniform((100, 140))  # dummy data
y = tf.reshape(tf.cast((x[:, 0] < 0.5), x.dtype), (-1, 1))  # dummy labels
y = tf.concat((x, y), axis=-1)  # dummy data + labels

def loss_fn(true, pred):
    def L_FN(true, pred):
        return tf.abs(tf.subtract(true, pred))

    true_label = true[:, -1:]  # separation
    true_x = true[:, :-1]
    fn = L_FN(true_x, pred)
    loss1 = tf.multiply(true_label, fn)
    loss2 = tf.multiply(tf.subtract(1.0, true_label), tf.math.log(tf.subtract(1.0, tf.math.exp(-fn))))
    loss = tf.subtract(loss1, loss2)
    return loss

autoencoder = AnomalyDetector()  # needs your AE code
autoencoder.compile('sgd', loss_fn)

In the loss function, the true prediction and the label gets separated again. Because the label is only in y (or true), the model prediction is not affected. The loss function should be the one from the paper (AE for the L_FN(x), not DAE).

2
  • Why true_label = true[:, -1:] instead of true_label = true[-1:] ? Commented Feb 22 at 19:24
  • The shape of true_label is (batch_size, labels). We want all from the batches, but only the last label, the added one. : is short for "take everything from this axis". In other words, it means "from all batches, take the last label". -1: also preserves the axis, because it is a slice from the last element to the end (which is just one one this axis). -1 (without the : after) would take the same element, but the axis would be gone. I hope I could make it clear. You can also take a look into array slicing.
    – mhenning
    Commented Feb 23 at 10:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.