NLLLoss

examples

m = nn.LogSoftmax(dim=1) loss = nn.NLLLoss() # input is of size N x C = 3 x 5 input = torch.randn(3, 5, requires_grad=True) # each element in target has to have 0 <= value < C target = torch.tensor([1, 0, 4]) output = loss(m(input), target) output.backward() # 2D loss example (used, for example, with image inputs) N, C = 5, 4 loss = nn.NLLLoss() # input is of size N x C x height x width data = torch.randn(N, 16, 10, 10) conv = nn.Conv2d(16, C, (3, 3)) m = nn.LogSoftmax(dim=1) # each element in target has to have 0 <= value < C target = torch.empty(N, 8, 8, dtype=torch.long).random_(0, C) output = loss(m(conv(data)), target) output.backward()

loss

unreduced

reduced

N是batchsize

理解

来自 https://www.cnblogs.com/ZJUT-jiangnan/p/5489047.html

NLLLoss前面必须要有log_softmax。拿着前面log_softmax输出的一批logy（这里的notation和上面的不一样。上面的y相当于这里的t，这里的logy相当于上面的x），找出target对应的那个logy，给它加个负号就是loss了。

CrossEntropyLoss

This criterion combines nn.LogSoftmax() and nn.NLLLoss() in one single class.

example

loss = nn.CrossEntropyLoss() input = torch.randn(3, 5, requires_grad=True) target = torch.empty(3, dtype=torch.long).random_(5) output = loss(input, target) output.backward()

loss

理解

跟上面NLLLoss里cross entropy = log_softmax + NLLLoss的搭配是一致的。

就是对每一个样本预测多个值，每个值对应一个类别，经过log_softmax后取target 类别对应的那个值。

BCELoss

example

m = nn.Sigmoid() loss = nn.BCELoss() input = torch.randn(3, requires_grad=True) target = torch.empty(3).random_(2) output = loss(m(input), target) output.backward()

loss

理解

前面一定要加sigmoid。每个样本预测一个值，经过sigmoid后加个log优化。

BCEWithLogitsLoss

就是上面那个把sigmoid层放进去，为了数值稳定。