in the pytorch NLLLoss doc the default of ignore_index is -100 instead of the usual None
, are there any particular reasons? seems like any negative value is equivalent.
BTW, what may be the reason that I would want to ignore an index? Thanks!
in the pytorch NLLLoss doc the default of ignore_index is -100 instead of the usual None
, are there any particular reasons? seems like any negative value is equivalent.
BTW, what may be the reason that I would want to ignore an index? Thanks!
The value for ignore_index
must be an int, that's why the default value is an int and not None
. The default value is arbitrary, it could have been any negative number, i.e. anything that is not a "valid" class label. The function will ignore all elements for which the target instance has that class label. In practice, this option can be used to identify unlabeled pixels for example in dense prediction tasks.
Edit: Tracing back the implementation of nn.NLLLoss
, we can find this comment in the nll_loss
implementation of torch/onnx/symbolic_opset12.py
:
# in onnx NegativeLogLikelihoodLoss specification, ignore_index is optional without default value.
# therefore we need to set ignore_index attribute even if it is not specified (e.g. ignore_index=-100).
ignore_index = sym_help._maybe_get_const(ignore_index, "i")
I feel that they should have used None
as a default. Python is not statically typed, so there is no need for ignore_index
to be an int (And even statically typed languages like C++ have option types these days) Failing that, they could have used -1
, which is less jarring.
So why didn't they use -1
, at least?
One possibility is that the authors were afraid that -1
would be misinterpreted as "the last one". I was able to find that the -100
was inherited from the Lua version of Torch. Although in Lua, -1
doesn't generally mean "last", Lua Torch's Tensor
class used this convention.