holocron.models¶
The models subpackage contains definitions of models for addressing different tasks, including: image classification, pixelwise semantic segmentation, object detection, instance segmentation, person keypoint detection and video classification.
Classification¶
Classification models expect a 4D image tensor as an input (N x C x H x W) and returns a 2D output (N x K). The output represents the classification scores for each output classes.
import holocron.models as models
darknet19 = models.darknet19(num_classes=10)
ResNet¶
- holocron.models.resnet18(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
ResNet-18 from “Deep Residual Learning for Image Recognition”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.resnet34(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
ResNet-34 from “Deep Residual Learning for Image Recognition”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.resnet50(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
ResNet-50 from “Deep Residual Learning for Image Recognition”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.resnet101(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
ResNet-101 from “Deep Residual Learning for Image Recognition”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.resnet152(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
ResNet-152 from “Deep Residual Learning for Image Recognition”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.resnext50_32x4d(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
ResNeXt-50 from “Aggregated Residual Transformations for Deep Neural Networks”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.resnext101_32x8d(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
ResNeXt-101 from “Aggregated Residual Transformations for Deep Neural Networks”
- Parameters:
- Returns:
classification model
- Return type:
Res2Net¶
TridentNet¶
PyConvResNet¶
- holocron.models.pyconv_resnet50(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
PyConvResNet-50 from “Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition”
- Parameters:
- Returns:
classification model
- Return type:
ReXNet¶
- holocron.models.rexnet1_0x(pretrained: bool = False, progress: bool = True, **kwargs: Any) ReXNet[source]¶
ReXNet-1.0x from “ReXNet: Diminishing Representational Bottleneck on Convolutional Neural Network”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.rexnet1_3x(pretrained: bool = False, progress: bool = True, **kwargs: Any) ReXNet[source]¶
ReXNet-1.3x from “ReXNet: Diminishing Representational Bottleneck on Convolutional Neural Network”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.rexnet1_5x(pretrained: bool = False, progress: bool = True, **kwargs: Any) ReXNet[source]¶
ReXNet-1.5x from “ReXNet: Diminishing Representational Bottleneck on Convolutional Neural Network”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.rexnet2_0x(pretrained: bool = False, progress: bool = True, **kwargs: Any) ReXNet[source]¶
ReXNet-2.0x from “ReXNet: Diminishing Representational Bottleneck on Convolutional Neural Network”
- Parameters:
- Returns:
classification model
- Return type:
SKNet¶
- holocron.models.sknet50(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
SKNet-50 from “Selective Kernel Networks”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.sknet101(pretrained: bool = False, progress: bool = True, **kwargs: Any) ResNet[source]¶
SKNet-101 from “Selective Kernel Networks”
- Parameters:
- Returns:
classification model
- Return type:
Darknet¶
- holocron.models.darknet24(pretrained: bool = False, progress: bool = True, **kwargs: Any) DarknetV1[source]¶
Darknet-24 from “You Only Look Once: Unified, Real-Time Object Detection”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.darknet19(pretrained: bool = False, progress: bool = True, **kwargs: Any) DarknetV2[source]¶
Darknet-19 from “YOLO9000: Better, Faster, Stronger”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.darknet53(pretrained: bool = False, progress: bool = True, **kwargs: Any) DarknetV3[source]¶
Darknet-53 from “YOLOv3: An Incremental Improvement”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.cspdarknet53(pretrained: bool = False, progress: bool = True, **kwargs: Any) DarknetV4[source]¶
CSP-Darknet-53 from “CSPNet: A New Backbone that can Enhance Learning Capability of CNN”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.cspdarknet53_mish(pretrained: bool = False, progress: bool = True, **kwargs: Any) DarknetV4[source]¶
Modified version of CSP-Darknet-53 from “CSPNet: A New Backbone that can Enhance Learning Capability of CNN” with Mish as activation layer and DropBlock as regularization layer.
- Parameters:
- Returns:
classification model
- Return type:
RepVGG¶
- holocron.models.repvgg_a0(pretrained: bool = False, progress: bool = True, **kwargs: Any) RepVGG[source]¶
RepVGG-A0 from “RepVGG: Making VGG-style ConvNets Great Again”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.repvgg_a1(pretrained: bool = False, progress: bool = True, **kwargs: Any) RepVGG[source]¶
RepVGG-A1 from “RepVGG: Making VGG-style ConvNets Great Again”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.repvgg_a2(pretrained: bool = False, progress: bool = True, **kwargs: Any) RepVGG[source]¶
RepVGG-A2 from “RepVGG: Making VGG-style ConvNets Great Again”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.repvgg_b0(pretrained: bool = False, progress: bool = True, **kwargs: Any) RepVGG[source]¶
RepVGG-B0 from “RepVGG: Making VGG-style ConvNets Great Again”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.repvgg_b1(pretrained: bool = False, progress: bool = True, **kwargs: Any) RepVGG[source]¶
RepVGG-B1 from “RepVGG: Making VGG-style ConvNets Great Again”
- Parameters:
- Returns:
classification model
- Return type:
- holocron.models.repvgg_b2(pretrained: bool = False, progress: bool = True, **kwargs: Any) RepVGG[source]¶
RepVGG-B2 from “RepVGG: Making VGG-style ConvNets Great Again”
- Parameters:
- Returns:
classification model
- Return type:
Object Detection¶
Object detection models expect a 4D image tensor as an input (N x C x H x W) and returns a list of dictionaries. Each dictionary has 3 keys: box coordinates, classification probability, classification label.
import holocron.models as models
yolov2 = models.yolov2(num_classes=10)
YOLO¶
- holocron.models.detection.yolov1(pretrained: bool = False, progress: bool = True, pretrained_backbone: bool = True, **kwargs: Any) YOLOv1[source]¶
YOLO model from “You Only Look Once: Unified, Real-Time Object Detection”.
YOLO’s particularity is to make predictions in a grid (same size as last feature map). For each grid cell, the model predicts classification scores and a fixed number of boxes (default: 2). Each box in the cell gets 5 predictions: an objectness score, and 4 coordinates. The 4 coordinates are composed of: the 2-D coordinates of the predicted box center (relative to the cell), and the width and height of the predicted box (relative to the whole image).
For training, YOLO uses a multi-part loss whose components are computed by:
\[\mathcal{L}_{coords} = \sum\limits_{i=0}^{S^2} \sum\limits_{j=0}^{B} \mathbb{1}_{ij}^{obj} \Big[ (x_{ij} - \hat{x}_{ij})² + (y_{ij} - \hat{y}_{ij})² + (\sqrt{w_{ij}} - \sqrt{\hat{w}_{ij}})² + (\sqrt{h_{ij}} - \sqrt{\hat{h}_{ij}})² \Big]\]where \(S\) is size of the output feature map (7 for an input size \((448, 448)\)), \(B\) is the number of anchor boxes per grid cell (default: 2), \(\mathbb{1}_{ij}^{obj}\) equals to 1 if a GT center falls inside the i-th grid cell and among the anchor boxes of that cell, has the highest IoU with the j-th box else 0, \((x_{ij}, y_{ij}, w_{ij}, h_{ij})\) are the coordinates of the ground truth assigned to the j-th anchor box of the i-th grid cell, and \((\hat{x}_{ij}, \hat{y}_{ij}, \hat{w}_{ij}, \hat{h}_{ij})\) are the coordinate predictions for the j-th anchor box of the i-th grid cell.
\[\mathcal{L}_{objectness} = \sum\limits_{i=0}^{S^2} \sum\limits_{j=0}^{B} \Big[ \mathbb{1}_{ij}^{obj} \Big(C_{ij} - \hat{C}_{ij} \Big)^2 + \lambda_{noobj} \mathbb{1}_{ij}^{noobj} \Big(C_{ij} - \hat{C}_{ij} \Big)^2 \Big]\]where \(\lambda_{noobj}\) is a positive coefficient (default: 0.5), \(\mathbb{1}_{ij}^{noobj} = 1 - \mathbb{1}_{ij}^{obj}\), \(C_{ij}\) equals the Intersection Over Union between the j-th anchor box in the i-th grid cell and its matched ground truth box if that box is matched with a ground truth else 0, and \(\hat{C}_{ij}\) is the objectness score of the j-th anchor box in the i-th grid cell..
\[\mathcal{L}_{classification} = \sum\limits_{i=0}^{S^2} \mathbb{1}_{i}^{obj} \sum\limits_{c \in classes} (p_i(c) - \hat{p}_i(c))^2\]where \(\mathbb{1}_{i}^{obj}\) equals to 1 if a GT center falls inside the i-th grid cell else 0, \(p_i(c)\) equals 1 if the assigned ground truth to the i-th cell is classified as class \(c\), and \(\hat{p}_i(c)\) is the predicted probability of class \(c\) in the i-th cell.
And the full loss is given by:
\[\mathcal{L}_{YOLOv1} = \lambda_{coords} \cdot \mathcal{L}_{coords} + \mathcal{L}_{objectness} + \mathcal{L}_{classification}\]where \(\lambda_{coords}\) is a positive coefficient (default: 5).
- Parameters:
- Returns:
detection module
- Return type:
- holocron.models.detection.yolov2(pretrained: bool = False, progress: bool = True, pretrained_backbone: bool = True, **kwargs: Any) YOLOv2[source]¶
YOLOv2 model from “YOLO9000: Better, Faster, Stronger”.
YOLOv2 improves upon YOLO by raising the number of boxes predicted by grid cell (default: 5), introducing bounding box priors and predicting class scores for each anchor box in the grid cell.
For training, YOLOv2 uses the same multi-part loss as YOLO apart from its classification loss:
\[\mathcal{L}_{classification} = \sum\limits_{i=0}^{S^2} \sum\limits_{j=0}^{B} \mathbb{1}_{ij}^{obj} \sum\limits_{c \in classes} (p_{ij}(c) - \hat{p}_{ij}(c))^2\]where \(S\) is size of the output feature map (13 for an input size \((416, 416)\)), \(B\) is the number of anchor boxes per grid cell (default: 5), \(\mathbb{1}_{ij}^{obj}\) equals to 1 if a GT center falls inside the i-th grid cell and among the anchor boxes of that cell, has the highest IoU with the j-th box else 0, \(p_{ij}(c)\) equals 1 if the assigned ground truth to the j-th anchor box of the i-th cell is classified as class \(c\), and \(\hat{p}_{ij}(c)\) is the predicted probability of class \(c\) for the j-th anchor box in the i-th cell.
- Parameters:
- Returns:
detection module
- Return type:
- holocron.models.detection.yolov4(pretrained: bool = False, progress: bool = True, pretrained_backbone: bool = True, **kwargs: Any) YOLOv4[source]¶
YOLOv4 model from “YOLOv4: Optimal Speed and Accuracy of Object Detection”.
The architecture improves upon YOLOv3 by including: the usage of DropBlock regularization, Mish activation, CSP and SAM in the backbone, SPP and PAN in the neck.
For training, YOLOv4 uses the same multi-part loss as YOLOv3 apart from its box coordinate loss:
\[\mathcal{L}_{coords} = \sum\limits_{i=0}^{S^2} \sum\limits_{j=0}^{B} \min\limits_{k \in [1, M]} C_{IoU}(\hat{loc}_{ij}, loc^{GT}_k)\]where \(S\) is size of the output feature map (13 for an input size \((416, 416)\)), \(B\) is the number of anchor boxes per grid cell (default: 3), \(M\) is the number of ground truth boxes, \(C_{IoU}\) is the complete IoU loss, \(\hat{loc}_{ij}\) is the predicted bounding box for grid cell \(i\) at anchor \(j\), and \(loc^{GT}_k\) is the k-th ground truth bounding box.
- Parameters:
- Returns:
detection module
- Return type:
Semantic Segmentation¶
Semantic segmentation models expect a 4D image tensor as an input (N x C x H x W) and returns a classification score tensor of size (N x K x Ho x Wo).
import holocron.models as models
unet = models.unet(num_classes=10)
U-Net¶
- holocron.models.segmentation.unet(pretrained: bool = False, progress: bool = True, **kwargs: Any) UNet[source]¶
U-Net from “U-Net: Convolutional Networks for Biomedical Image Segmentation”
- Parameters:
pretrained – If True, returns a model pre-trained on PASCAL VOC2012
progress – If True, displays a progress bar of the download to stderr
- Returns:
semantic segmentation model
- holocron.models.segmentation.unetp(pretrained: bool = False, progress: bool = True, **kwargs: Any) UNetp[source]¶
UNet+ from “UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation”
- Parameters:
pretrained – If True, returns a model pre-trained on PASCAL VOC2012
progress – If True, displays a progress bar of the download to stderr
- Returns:
semantic segmentation model
- holocron.models.segmentation.unetpp(pretrained: bool = False, progress: bool = True, **kwargs: Any) UNetpp[source]¶
UNet++ from “UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation”
- Parameters:
pretrained – If True, returns a model pre-trained on PASCAL VOC2012
progress – If True, displays a progress bar of the download to stderr
- Returns:
semantic segmentation model
- holocron.models.segmentation.unet3p(pretrained: bool = False, progress: bool = True, **kwargs: Any) UNet3p[source]¶
UNet3+ from “UNet 3+: A Full-Scale Connected UNet For Medical Image Segmentation”
- Parameters:
pretrained – If True, returns a model pre-trained on PASCAL VOC2012
progress – If True, displays a progress bar of the download to stderr
- Returns:
semantic segmentation model
- holocron.models.segmentation.unet2(pretrained: bool = False, progress: bool = True, in_channels: int = 3, **kwargs: Any) DynamicUNet[source]¶
Modified version of U-Net from “U-Net: Convolutional Networks for Biomedical Image Segmentation” that includes a more advanced upscaling block inspired by fastai.
- Parameters:
pretrained – If True, returns a model pre-trained on PASCAL VOC2012
progress – If True, displays a progress bar of the download to stderr
in_channels – number of input channels
- Returns:
semantic segmentation model
- holocron.models.segmentation.unet_tvvgg11(pretrained: bool = False, pretrained_backbone: bool = True, progress: bool = True, **kwargs: Any) DynamicUNet[source]¶
U-Net from “U-Net: Convolutional Networks for Biomedical Image Segmentation” with a VGG-11 backbone used as encoder, and more advanced upscaling blocks inspired by fastai.
- Parameters:
pretrained – If True, returns a model pre-trained on PASCAL VOC2012
pretrained_backbone – If True, the encoder will load pretrained parameters from ImageNet
progress – If True, displays a progress bar of the download to stderr
- Returns:
semantic segmentation model
- holocron.models.segmentation.unet_tvresnet34(pretrained: bool = False, pretrained_backbone: bool = True, progress: bool = True, **kwargs: Any) DynamicUNet[source]¶
U-Net from “U-Net: Convolutional Networks for Biomedical Image Segmentation” with a ResNet-34 backbone used as encoder, and more advanced upscaling blocks inspired by fastai.
- Parameters:
pretrained – If True, returns a model pre-trained on PASCAL VOC2012
pretrained_backbone – If True, the encoder will load pretrained parameters from ImageNet
progress – If True, displays a progress bar of the download to stderr
- Returns:
semantic segmentation model
- holocron.models.segmentation.unet_rexnet13(pretrained: bool = False, pretrained_backbone: bool = True, progress: bool = True, in_channels: int = 3, **kwargs: Any) DynamicUNet[source]¶
U-Net from “U-Net: Convolutional Networks for Biomedical Image Segmentation” with a ReXNet-1.3x backbone used as encoder, and more advanced upscaling blocks inspired by fastai.
- Parameters:
pretrained – If True, returns a model pre-trained on PASCAL VOC2012
pretrained_backbone – If True, the encoder will load pretrained parameters from ImageNet
progress – If True, displays a progress bar of the download to stderr
- Returns:
semantic segmentation model