When I looked the example code of BCEWithLogitsLoss in PyTorch Docs. I am confusing about the output result of loss function and formula.
>>> loss = nn.BCEWithLogitsLoss()
>>> input = torch.randn(3, requires_grad=True)
>>> target = torch.empty(3).random_(2)
>>> output = loss(input, target)
>>> output.backward()
input : tensor([0.4764, -2.4063, 0.1563], requires_grad=True)
target: tensor([0., 1., 1.])
output: tensor(1.3567, grad_fn=<BinaryCrossEntropyWithLogitsBackward>)
But according to the formula by showing:
The output of loss function should have shape (3,) instead of single value ,since the shape of input and output both are (3,) . I was thinking that the output may was the sum of Ln or else, but still no idea. Could someone help me to explain that?
As @Dishin H Goyani reminded that the default reduction is 'mean'. I did a simple test.
>>> target_n = target.numpy()
>>> input_n = input.detach().numpy()
>>> def sigmoid(array):return 1/(1+np.exp(-array))
>>> output_n = -1*(target_n*np.log(sigmoid(input_n))+(1-target_n)*np.log(1-sigmoid(input_n)))
output_n : array([0.95947516, 2.4926252 , 0.61806685], dtype=float32)
>>> np.mean(output_n)
1.3567224
The result is matched.
As you seen, the default Wn is 1.
reduction='none'