Softmax：
$\sigma(z)_j = \frac{e^{z_j}}{\sum_1^k e^{z_k}}$
Log-softmax:
$\sigma(z)_j = z_j-\log{\sum_1^k e^{z_k}}$
In PyTorch:
nn.CrossEntropyLoss is equivalent to the combination of F.log_softmax and F.nll_loss
F.nll_loss only computes $\sum_i -y_i$ (depends on reduction).
PREVIOUSLinux