nn.BCEWithLogitsLoss
is actually just cross entropy loss that comes inside a sigmoid function. It may be used in case your model's output layer is not wrapped with sigmoid. Typically used with the raw output of a single output layer neuron.
Simply put, your model's output say pred
will be a raw value. In order to get probability, you will have to use torch.sigmoid(pred)
. (To get actual class labels, you need torch.round(torch.sigmoid(pred))
.) However, you don't need to do anything like that (i.e take sigmoid) when you use nn.BCEWithLogitsLoss
. Here you just have to do the following-
criterion = nn.BCEWithLogitsLoss()
loss = criterion(pred, target) # pred is just raw nn output
Hence, coming to implementation part, criterion accepts two torch tensors - one being the raw nn outputs, the other being the true class labels, then wraps the first using sigmoid - for each element in the tensor and then calculates Cross Entropy loss (-(target*log(sigmoid(pred)))
for each pair and reduces it to mean.