holocron.nn

An addition to the torch.nn module of Pytorch to extend the range of neural networks building blocks.

Non-linear activations

class holocron.nn.Mish

Implements the Mish activation module from “Mish: A Self Regularized Non-Monotonic Neural Activation Function”

class holocron.nn.NLReLU(inplace=False)

Implements the Natural-Logarithm ReLU activation module from “Natural-Logarithm-Rectified Activation Function in Convolutional Neural Networks”

Parameters:

inplace (bool) – should the operation be performed inplace

Loss functions

class holocron.nn.FocalLoss(gamma=2, **kwargs)

Implementation of Focal Loss as described in “Focal Loss for Dense Object Detection”

Parameters:

• gamma (float, optional) – exponent parameter of the focal loss

• weight (torch.Tensor[K], optional) – class weight for loss computation

• ignore_index (int, optional) – specifies target value that is ignored and do not contribute to gradient

• reduction (str, optional) – type of reduction to apply to the final loss

class holocron.nn.MultiLabelCrossEntropy(**kwargs)

Implementation of the cross-entropy loss for multi-label targets

Parameters:

• weight (torch.Tensor[K], optional) – class weight for loss computation

• ignore_index (int, optional) – specifies target value that is ignored and do not contribute to gradient

• reduction (str, optional) – type of reduction to apply to the final loss

class holocron.nn.LabelSmoothingCrossEntropy(eps=0.1, **kwargs)

Implementation of the cross-entropy loss with label smoothing on hard target as described in “Attention Is All You Need”

Parameters:

• eps (float, optional) – smoothing factor

• weight (torch.Tensor[K], optional) – class weight for loss computation

• ignore_index (int, optional) – specifies target value that is ignored and do not contribute to gradient

• reduction (str, optional) – type of reduction to apply to the final loss

Loss wrappers

class holocron.nn.MixupLoss(criterion)

Implements a Mixup wrapper as described in “mixup: Beyond Empirical Risk Minimization”

Parameters:

criterion (callable) – initial criterion to be used on normal sample & targets

Downsampling

class holocron.nn.ConcatDownsample2d(scale_factor)

Implements a loss-less downsampling operation described in “YOLO9000: Better, Faster, Stronger” by stacking adjacent information on the channel dimension.

Parameters:

scale_factor (int) – spatial scaling factor