holocron.nn.functional

Non-linear activations

holocron.nn.functional.silu(x)[source]

Implements the SiLU activation function

Parameters:

x (torch.Tensor) – input tensor

Returns:

output tensor

Return type:

torch.Tensor[x.size()]

holocron.nn.functional.mish(x)[source]

Implements the Mish activation function

Parameters:

x (torch.Tensor) – input tensor

Returns:

output tensor

Return type:

torch.Tensor[x.size()]

holocron.nn.functional.nl_relu(x, beta=1.0, inplace=False)[source]

Implements the natural logarithm ReLU activation function

Parameters:

  • x (torch.Tensor) – input tensor

  • beta (float) – beta used for NReLU

  • inplace (bool) – whether the operation should be performed inplace

Returns:

output tensor

Return type:

torch.Tensor[x.size()]

Loss functions

holocron.nn.functional.focal_loss(x, target, weight=None, ignore_index=-100, reduction='mean', gamma=2)[source]

Implements the focal loss from “Focal Loss for Dense Object Detection”

Parameters:

  • x (torch.Tensor[N, K, ...]) – input tensor

  • target (torch.Tensor[N, ...]) – hard target tensor

  • weight (torch.Tensor[K], optional) – manual rescaling of each class

  • ignore_index (int, optional) – specifies target value that is ignored and do not contribute to gradient

  • reduction (str, optional) – reduction method

  • gamma (float, optional) – gamma parameter of focal loss

Returns:

loss reduced with reduction method

Return type:

torch.Tensor

holocron.nn.functional.multilabel_cross_entropy(x, target, weight=None, ignore_index=-100, reduction='mean')[source]

Implements the cross entropy loss for multi-label targets

Parameters:

  • x (torch.Tensor[N, K, ...]) – input tensor

  • target (torch.Tensor[N, K, ...]) – target tensor

  • weight (torch.Tensor[K], optional) – manual rescaling of each class

  • ignore_index (int, optional) – specifies target value that is ignored and do not contribute to gradient

  • reduction (str, optional) – reduction method

Returns:

loss reduced with reduction method

Return type:

torch.Tensor

holocron.nn.functional.ls_cross_entropy(x, target, weight=None, ignore_index=-100, reduction='mean', eps=0.1)[source]

Implements the label smoothing cross entropy loss from “Attention Is All You Need”

Parameters:

  • x (torch.Tensor[N, K, ...]) – input tensor

  • target (torch.Tensor[N, ...]) – target tensor

  • weight (torch.Tensor[K], optional) – manual rescaling of each class

  • ignore_index (int, optional) – specifies target value that is ignored and do not contribute to gradient

  • reduction (str, optional) – reduction method

  • eps (float, optional) – smoothing factor

Returns:

loss reduced with reduction method

Return type:

torch.Tensor

Convolutions

holocron.nn.functional.norm_conv2d(x, weight, bias=None, stride=1, padding=0, dilation=1, groups=1, eps=1e-14)[source]

Implements a normalized convolution operations in 2D. Based on the implementation by the paper’s author. See NormConv2d for details and output shape.

Parameters:

  • x (torch.Tensor[N, in_channels, H, W]) – input tensor

  • weight (torch.Tensor[out_channels, in_channels, Kh, Kw]) – filters

  • bias (torch.Tensor[out_channels], optional) – optional bias tensor of shape (out_channels). Default: None

  • stride (int, optional) – the stride of the convolving kernel. Can be a single number or a tuple (sH, sW). Default: 1

  • padding (int, optional) – implicit paddings on both sides of the input. Can be a single number or a tuple (padH, padW). Default: 0

  • dilation (int, optional) – the spacing between kernel elements. Can be a single number or a tuple (dH, dW). Default: 1

  • groups (int, optional) – split input into groups, in_channels should be divisible by the number of groups. Default: 1

  • eps (float, optional) – a value added to the denominator for numerical stability. Default: 1e-14

Examples::
>>> # With square kernels and equal stride
>>> filters = torch.randn(8,4,3,3)
>>> inputs = torch.randn(1,4,5,5)
>>> F.norm_conv2d(inputs, filters, padding=1)
holocron.nn.functional.add2d(x, weight, bias=None, stride=1, padding=0, dilation=1, groups=1, normalize_slices=False, eps=1e-14)[source]

Implements an adder operation in 2D from “AdderNet: Do We Really Need Multiplications in Deep Learning?”. See Add2d for details and output shape.

Parameters:

  • x (torch.Tensor[N, in_channels, H, W]) – input tensor

  • weight (torch.Tensor[out_channels, in_channels, Kh, Kw]) – filters

  • bias (torch.Tensor[out_channels], optional) – optional bias tensor of shape (out_channels). Default: None

  • stride (int, optional) – the stride of the convolving kernel. Can be a single number or a tuple (sH, sW). Default: 1

  • padding (int, optional) – implicit paddings on both sides of the input. Can be a single number or a tuple (padH, padW). Default: 0

  • dilation (int, optional) – the spacing between kernel elements. Can be a single number or a tuple (dH, dW). Default: 1

  • groups (int, optional) – split input into groups, in_channels should be divisible by the number of groups. Default: 1

  • normalize_slices (bool, optional) – whether input slices should be normalized

  • eps (float, optional) – a value added to the denominator for numerical stability. Default: 1e-14

Examples::
>>> # With square kernels and equal stride
>>> filters = torch.randn(8,4,3,3)
>>> inputs = torch.randn(1,4,5,5)
>>> F.norm_conv2d(inputs, filters, padding=1)

Regularization layers

holocron.nn.functional.dropblock2d(x, drop_prob, block_size, inplace=False)[source]

Implements the dropblock operation from “DropBlock: A regularization method for convolutional networks”

Parameters:

  • drop_prob (float) – probability of dropping activation value

  • block_size (int) – size of each block that is expended from the sampled mask

  • inplace (bool, optional) – whether the operation should be done inplace

Downsampling

holocron.nn.functional.concat_downsample2d(x, scale_factor)[source]

Implements a loss-less downsampling operation described in “YOLO9000: Better, Faster, Stronger” by stacking adjacent information on the channel dimension.

Parameters:

  • x (torch.Tensor[N, C, H, W]) – input tensor

  • scale_factor (int) – spatial scaling factor

Returns:

downsampled tensor

Return type:

torch.Tensor[N, scale_factor ** 2 * C, H / scale_factor, W / scale_factor]