Skip to content

RepVGG

The ResNet model is based on the "RepVGG: Making VGG-style ConvNets Great Again" paper.

Architecture overview

This paper revisits the VGG architecture by adapting its parameter setting in training and inference mode to combine the original VGG speed and the block design of ResNet.

RepVGG architecture

The key takeaways from the paper are the following:

  • have different block architectures between training and inference modes
  • the block is designed in a similar fashion as a ResNet bottleneck but in a way that all branches can be fused into a single one
  • The more complex training architecture improves gradient flow and overall optimization, while its inference counterpart is optimized for minimum latency and memory usage

Model builders

The following model builders can be used to instantiate a RepVGG model, with or without pre-trained weights. All the model builders internally rely on the RepVGG base class.

RepVGG 

RepVGG(num_blocks: list[int], planes: list[int], width_multiplier: float, final_width_multiplier: float, num_classes: int = 10, in_channels: int = 3, act_layer: Module | None = None, norm_layer: Callable[[int], Module] | None = None)

Bases: Sequential

Implements a reparametrized version of VGG as described in "RepVGG: Making VGG-style ConvNets Great Again" <https://arxiv.org/pdf/2101.03697.pdf>_

PARAMETER DESCRIPTION
num_blocks

list of number of blocks per stage

TYPE: list[int]

planes

list of output channels of each stage

TYPE: list[int]

width_multiplier

multiplier for the output channels of all stages apart from the last

TYPE: float

final_width_multiplier

multiplier for the output channels of the last stage

TYPE: float

num_classes

number of output classes

TYPE: int DEFAULT: 10

in_channels

number of input channels

TYPE: int DEFAULT: 3

act_layer

the activation layer to use

TYPE: Module | None DEFAULT: None

norm_layer

the normalization layer to use

TYPE: Callable[[int], Module] | None DEFAULT: None

Source code in holocron/models/classification/repvgg.py 
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
def __init__(
    self,
    num_blocks: list[int],
    planes: list[int],
    width_multiplier: float,
    final_width_multiplier: float,
    num_classes: int = 10,
    in_channels: int = 3,
    act_layer: nn.Module | None = None,
    norm_layer: Callable[[int], nn.Module] | None = None,
) -> None:
    if norm_layer is None:
        norm_layer = nn.BatchNorm2d
    if act_layer is None:
        act_layer = nn.ReLU(inplace=True)

    if len(num_blocks) != len(planes):
        raise AssertionError("the length of `num_blocks` and `planes` are expected to be the same")

    stages: list[nn.Sequential] = []
    # Assign the width multipliers
    chans = [in_channels, int(min(1, width_multiplier) * planes[0])]
    chans.extend([int(width_multiplier * chan) for chan in planes[1:-1]])
    chans.append(int(final_width_multiplier * planes[-1]))

    # Build the layers
    for nb_blocks, in_chan, out_chan in zip(num_blocks, chans[:-1], chans[1:], strict=True):
        layers = [RepBlock(in_chan, out_chan, 2, False, act_layer, norm_layer)]
        layers.extend([RepBlock(out_chan, out_chan, 1, True, act_layer, norm_layer) for _ in range(nb_blocks)])
        stages.append(nn.Sequential(*layers))

    super().__init__(
        OrderedDict([
            ("features", nn.Sequential(*stages)),
            ("pool", GlobalAvgPool2d(flatten=True)),
            ("head", nn.Linear(chans[-1], num_classes)),
        ])
    )
    # Init all layers
    init.init_module(self, nonlinearity="relu")

reparametrize 

reparametrize() -> None

Reparametrize the block by fusing convolutions and BN in each branch, then fusing all branches

Source code in holocron/models/classification/repvgg.py 
171
172
173
174
175
176
def reparametrize(self) -> None:
    """Reparametrize the block by fusing convolutions and BN in each branch, then fusing all branches"""
    self.features: nn.Sequential
    for stage in self.features:
        for block in stage:
            block.reparametrize()

repvgg_a0 

repvgg_a0(pretrained: bool = False, checkpoint: Checkpoint | None = None, progress: bool = True, **kwargs: Any) -> RepVGG

RepVGG-A0 from "RepVGG: Making VGG-style ConvNets Great Again"

PARAMETER DESCRIPTION
pretrained

If True, returns a model pre-trained on ImageNette

TYPE: bool DEFAULT: False

checkpoint

If specified, the model's parameters will be set to the checkpoint's values

TYPE: Checkpoint | None DEFAULT: None

progress

If True, displays a progress bar of the download to stderr

TYPE: bool DEFAULT: True

kwargs

keyword args of RepVGG

TYPE: Any DEFAULT: {}

RETURNS DESCRIPTION
RepVGG

classification model

RepVGG_A0_Checkpoint

Bases: Enum

IMAGENETTE class-attribute instance-attribute 
IMAGENETTE = _checkpoint(arch='repvgg_a0', url='https://github.com/frgfm/Holocron/releases/download/v0.2.1/repvgg_a0_224-d3f54b28.pth', acc1=0.9292, acc5=0.9946, sha256='d3f54b28567fcd7e3e32ffbcffb5bb5c64fd97b7139cba0bfe9ad0bd7765cdaa', size=99183419, num_params=24741642, commit='d4a59999179b42fc0d3058ac6b76cc41f49dd56e', train_args='./imagenette2-320/ --arch repvgg_a0 --batch-size 64 --mixup-alpha 0.2 --amp --device 0 --epochs 100 --lr 1e-3 --label-smoothing 0.1 --random-erase 0.1 --train-crop-size 176 --val-resize-size 232 --opt adamw --weight-decay 5e-2')
DEFAULT class-attribute instance-attribute 
DEFAULT = IMAGENETTE
Source code in holocron/models/classification/repvgg.py 
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
def repvgg_a0(
    pretrained: bool = False,
    checkpoint: Checkpoint | None = None,
    progress: bool = True,
    **kwargs: Any,
) -> RepVGG:
    """RepVGG-A0 from
    ["RepVGG: Making VGG-style ConvNets Great Again"](https://arxiv.org/pdf/2101.03697.pdf)

    Args:
        pretrained: If True, returns a model pre-trained on ImageNette
        checkpoint: If specified, the model's parameters will be set to the checkpoint's values
        progress: If True, displays a progress bar of the download to stderr
        kwargs: keyword args of [`RepVGG`][holocron.models.classification.repvgg.RepVGG]

    Returns:
        classification model

    ::: holocron.models.RepVGG_A0_Checkpoint
        options:
            heading_level: 4
            show_if_no_docstring: true
    """
    checkpoint = _handle_legacy_pretrained(
        pretrained,
        checkpoint,
        RepVGG_A0_Checkpoint.DEFAULT.value,
    )
    return _repvgg(checkpoint, progress, [1, 2, 4, 14, 1], 0.75, 2.5, **kwargs)

repvgg_a1 

repvgg_a1(pretrained: bool = False, checkpoint: Checkpoint | None = None, progress: bool = True, **kwargs: Any) -> RepVGG

RepVGG-A1 from "RepVGG: Making VGG-style ConvNets Great Again"

PARAMETER DESCRIPTION
pretrained

If True, returns a model pre-trained on ImageNette

TYPE: bool DEFAULT: False

checkpoint

If specified, the model's parameters will be set to the checkpoint's values

TYPE: Checkpoint | None DEFAULT: None

progress

If True, displays a progress bar of the download to stderr

TYPE: bool DEFAULT: True

kwargs

keyword args of RepVGG

TYPE: Any DEFAULT: {}

RETURNS DESCRIPTION
RepVGG

classification model

RepVGG_A1_Checkpoint

Bases: Enum

IMAGENETTE class-attribute instance-attribute 
IMAGENETTE = _checkpoint(arch='repvgg_a1', url='https://github.com/frgfm/Holocron/releases/download/v0.2.1/repvgg_a1_224-8d3269fb.pth', acc1=0.9378, acc5=0.9918, sha256='8d3269fb5181c0fe75ef617872238135f3002f41e82e5ef7492d62a402ffae50', size=120724868, num_params=30119946, commit='d4a59999179b42fc0d3058ac6b76cc41f49dd56e', train_args='./imagenette2-320/ --arch repvgg_a1 --batch-size 64 --mixup-alpha 0.2 --amp --device 0 --epochs 100 --lr 1e-3 --label-smoothing 0.1 --random-erase 0.1 --train-crop-size 176 --val-resize-size 232 --opt adamw --weight-decay 5e-2')
DEFAULT class-attribute instance-attribute 
DEFAULT = IMAGENETTE
Source code in holocron/models/classification/repvgg.py 
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
def repvgg_a1(
    pretrained: bool = False,
    checkpoint: Checkpoint | None = None,
    progress: bool = True,
    **kwargs: Any,
) -> RepVGG:
    """RepVGG-A1 from
    ["RepVGG: Making VGG-style ConvNets Great Again"](https://arxiv.org/pdf/2101.03697.pdf)

    Args:
        pretrained: If True, returns a model pre-trained on ImageNette
        checkpoint: If specified, the model's parameters will be set to the checkpoint's values
        progress: If True, displays a progress bar of the download to stderr
        kwargs: keyword args of [`RepVGG`][holocron.models.classification.repvgg.RepVGG]

    Returns:
        classification model

    ::: holocron.models.RepVGG_A1_Checkpoint
        options:
            heading_level: 4
            show_if_no_docstring: true
    """
    checkpoint = _handle_legacy_pretrained(
        pretrained,
        checkpoint,
        RepVGG_A1_Checkpoint.DEFAULT.value,
    )
    return _repvgg(checkpoint, progress, [1, 2, 4, 14, 1], 1, 2.5, **kwargs)

repvgg_a2 

repvgg_a2(pretrained: bool = False, checkpoint: Checkpoint | None = None, progress: bool = True, **kwargs: Any) -> RepVGG

RepVGG-A2 from "RepVGG: Making VGG-style ConvNets Great Again"

PARAMETER DESCRIPTION
pretrained

If True, returns a model pre-trained on ImageNette

TYPE: bool DEFAULT: False

checkpoint

If specified, the model's parameters will be set to the checkpoint's values

TYPE: Checkpoint | None DEFAULT: None

progress

If True, displays a progress bar of the download to stderr

TYPE: bool DEFAULT: True

kwargs

keyword args of RepVGG

TYPE: Any DEFAULT: {}

RETURNS DESCRIPTION
RepVGG

classification model

RepVGG_A2_Checkpoint

Bases: Enum

IMAGENETTE class-attribute instance-attribute 
IMAGENETTE = _checkpoint(arch='repvgg_a2', url='https://github.com/frgfm/Holocron/releases/download/v0.2.1/repvgg_a2_224-cb442207.pth', acc1=0.9363, acc5=0.9939, sha256='cb442207d0c4627e3a16d7a8b4bf5342a182fd924cf4a044ac3a832014e7d4cf', size=194822538, num_params=48629514, commit='d4a59999179b42fc0d3058ac6b76cc41f49dd56e', train_args='./imagenette2-320/ --arch repvgg_a2 --batch-size 64 --mixup-alpha 0.2 --amp --device 0 --epochs 100 --lr 1e-3 --label-smoothing 0.1 --random-erase 0.1 --train-crop-size 176 --val-resize-size 232 --opt adamw --weight-decay 5e-2')
DEFAULT class-attribute instance-attribute 
DEFAULT = IMAGENETTE
Source code in holocron/models/classification/repvgg.py 
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
def repvgg_a2(
    pretrained: bool = False,
    checkpoint: Checkpoint | None = None,
    progress: bool = True,
    **kwargs: Any,
) -> RepVGG:
    """RepVGG-A2 from
    ["RepVGG: Making VGG-style ConvNets Great Again"](https://arxiv.org/pdf/2101.03697.pdf)

    Args:
        pretrained: If True, returns a model pre-trained on ImageNette
        checkpoint: If specified, the model's parameters will be set to the checkpoint's values
        progress: If True, displays a progress bar of the download to stderr
        kwargs: keyword args of [`RepVGG`][holocron.models.classification.repvgg.RepVGG]

    Returns:
        classification model

    ::: holocron.models.RepVGG_A2_Checkpoint
        options:
            heading_level: 4
            show_if_no_docstring: true
    """
    checkpoint = _handle_legacy_pretrained(
        pretrained,
        checkpoint,
        RepVGG_A2_Checkpoint.DEFAULT.value,
    )
    return _repvgg(checkpoint, progress, [1, 2, 4, 14, 1], 1.5, 2.75, **kwargs)

repvgg_b0 

repvgg_b0(pretrained: bool = False, checkpoint: Checkpoint | None = None, progress: bool = True, **kwargs: Any) -> RepVGG

RepVGG-B0 from "RepVGG: Making VGG-style ConvNets Great Again"

PARAMETER DESCRIPTION
pretrained

If True, returns a model pre-trained on ImageNette

TYPE: bool DEFAULT: False

checkpoint

If specified, the model's parameters will be set to the checkpoint's values

TYPE: Checkpoint | None DEFAULT: None

progress

If True, displays a progress bar of the download to stderr

TYPE: bool DEFAULT: True

kwargs

keyword args of RepVGG

TYPE: Any DEFAULT: {}

RETURNS DESCRIPTION
RepVGG

classification model

RepVGG_B0_Checkpoint

Bases: Enum

IMAGENETTE class-attribute instance-attribute 
IMAGENETTE = _checkpoint(arch='repvgg_b0', url='https://github.com/frgfm/Holocron/releases/download/v0.2.1/repvgg_b0_224-fdcdd2b7.pth', acc1=0.9269, acc5=0.9921, sha256='fdcdd2b739f19b47572be5a98ec407c08935d02adf1ab0bf90d7bc92c710fe2d', size=127668600, num_params=31845642, commit='d4a59999179b42fc0d3058ac6b76cc41f49dd56e', train_args='./imagenette2-320/ --arch repvgg_b0 --batch-size 64 --mixup-alpha 0.2 --amp --device 0 --epochs 100 --lr 1e-3 --label-smoothing 0.1 --random-erase 0.1 --train-crop-size 176 --val-resize-size 232 --opt adamw --weight-decay 5e-2')
DEFAULT class-attribute instance-attribute 
DEFAULT = IMAGENETTE
Source code in holocron/models/classification/repvgg.py 
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
def repvgg_b0(
    pretrained: bool = False,
    checkpoint: Checkpoint | None = None,
    progress: bool = True,
    **kwargs: Any,
) -> RepVGG:
    """RepVGG-B0 from
    ["RepVGG: Making VGG-style ConvNets Great Again"](https://arxiv.org/pdf/2101.03697.pdf)

    Args:
        pretrained: If True, returns a model pre-trained on ImageNette
        checkpoint: If specified, the model's parameters will be set to the checkpoint's values
        progress: If True, displays a progress bar of the download to stderr
        kwargs: keyword args of [`RepVGG`][holocron.models.classification.repvgg.RepVGG]

    Returns:
        classification model

    ::: holocron.models.RepVGG_B0_Checkpoint
        options:
            heading_level: 4
            show_if_no_docstring: true
    """
    checkpoint = _handle_legacy_pretrained(
        pretrained,
        checkpoint,
        RepVGG_B0_Checkpoint.DEFAULT.value,
    )
    return _repvgg(checkpoint, progress, [1, 4, 6, 16, 1], 1, 2.5, **kwargs)

repvgg_b1 

repvgg_b1(pretrained: bool = False, checkpoint: Checkpoint | None = None, progress: bool = True, **kwargs: Any) -> RepVGG

RepVGG-B1 from "RepVGG: Making VGG-style ConvNets Great Again"

PARAMETER DESCRIPTION
pretrained

If True, returns a model pre-trained on ImageNette

TYPE: bool DEFAULT: False

checkpoint

If specified, the model's parameters will be set to the checkpoint's values

TYPE: Checkpoint | None DEFAULT: None

progress

If True, displays a progress bar of the download to stderr

TYPE: bool DEFAULT: True

kwargs

keyword args of RepVGG

TYPE: Any DEFAULT: {}

RETURNS DESCRIPTION
RepVGG

classification model

RepVGG_B1_Checkpoint

Bases: Enum

IMAGENETTE class-attribute instance-attribute 
IMAGENETTE = _checkpoint(arch='repvgg_b1', url='https://github.com/frgfm/Holocron/releases/download/v0.2.1/repvgg_b1_224-3e5b28d7.pth', acc1=0.9396, acc5=0.9939, sha256='3e5b28d7803965546efadeb20abb84d8fef765dd08170677467a9c06294224c4', size=403763795, num_params=100829194, commit='d4a59999179b42fc0d3058ac6b76cc41f49dd56e', train_args='./imagenette2-320/ --arch repvgg_b1 --batch-size 64 --mixup-alpha 0.2 --amp --device 0 --epochs 100 --lr 1e-3 --label-smoothing 0.1 --random-erase 0.1 --train-crop-size 176 --val-resize-size 232 --opt adamw --weight-decay 5e-2')
DEFAULT class-attribute instance-attribute 
DEFAULT = IMAGENETTE
Source code in holocron/models/classification/repvgg.py 
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
def repvgg_b1(
    pretrained: bool = False,
    checkpoint: Checkpoint | None = None,
    progress: bool = True,
    **kwargs: Any,
) -> RepVGG:
    """RepVGG-B1 from
    ["RepVGG: Making VGG-style ConvNets Great Again"](https://arxiv.org/pdf/2101.03697.pdf)

    Args:
        pretrained: If True, returns a model pre-trained on ImageNette
        checkpoint: If specified, the model's parameters will be set to the checkpoint's values
        progress: If True, displays a progress bar of the download to stderr
        kwargs: keyword args of [`RepVGG`][holocron.models.RepVGG]

    Returns:
        classification model

    ::: holocron.models.RepVGG_B1_Checkpoint
        options:
            heading_level: 4
            show_if_no_docstring: true
    """
    checkpoint = _handle_legacy_pretrained(
        pretrained,
        checkpoint,
        RepVGG_B1_Checkpoint.DEFAULT.value,
    )
    return _repvgg(checkpoint, progress, [1, 4, 6, 16, 1], 2, 4, **kwargs)

repvgg_b2 

repvgg_b2(pretrained: bool = False, checkpoint: Checkpoint | None = None, progress: bool = True, **kwargs: Any) -> RepVGG

RepVGG-B2 from "RepVGG: Making VGG-style ConvNets Great Again"

PARAMETER DESCRIPTION
pretrained

If True, returns a model pre-trained on ImageNette

TYPE: bool DEFAULT: False

checkpoint

If specified, the model's parameters will be set to the checkpoint's values

TYPE: Checkpoint | None DEFAULT: None

progress

If True, displays a progress bar of the download to stderr

TYPE: bool DEFAULT: True

kwargs

keyword args of RepVGG

TYPE: Any DEFAULT: {}

RETURNS DESCRIPTION
RepVGG

classification model

RepVGG_B2_Checkpoint

Bases: Enum

IMAGENETTE class-attribute instance-attribute 
IMAGENETTE = _checkpoint(arch='repvgg_b2', url='https://github.com/frgfm/Holocron/releases/download/v0.2.1/repvgg_b2_224-dc810d88.pth', acc1=0.9414, acc5=0.9957, sha256='dc810d889e8533f3ab24d75d8bf4cec84380abfb3b10ee01009997eab6a35d4b', size=630382163, num_params=157462410, commit='d4a59999179b42fc0d3058ac6b76cc41f49dd56e', train_args='./imagenette2-320/ --arch repvgg_b2 --batch-size 32 --grad-acc 2 --mixup-alpha 0.2 --amp --device 0 --epochs 100 --lr 1e-3 --label-smoothing 0.1 --random-erase 0.1 --train-crop-size 176 --val-resize-size 232 --opt adamw --weight-decay 5e-2')
DEFAULT class-attribute instance-attribute 
DEFAULT = IMAGENETTE
Source code in holocron/models/classification/repvgg.py 
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
def repvgg_b2(
    pretrained: bool = False,
    checkpoint: Checkpoint | None = None,
    progress: bool = True,
    **kwargs: Any,
) -> RepVGG:
    """RepVGG-B2 from
    ["RepVGG: Making VGG-style ConvNets Great Again"](https://arxiv.org/pdf/2101.03697.pdf)

    Args:
        pretrained: If True, returns a model pre-trained on ImageNette
        checkpoint: If specified, the model's parameters will be set to the checkpoint's values
        progress: If True, displays a progress bar of the download to stderr
        kwargs: keyword args of [`RepVGG`][holocron.models.classification.repvgg.RepVGG]

    Returns:
        classification model

    ::: holocron.models.RepVGG_B2_Checkpoint
        options:
            heading_level: 4
            show_if_no_docstring: true
    """
    checkpoint = _handle_legacy_pretrained(
        pretrained,
        checkpoint,
        RepVGG_B2_Checkpoint.DEFAULT.value,
    )
    return _repvgg(checkpoint, progress, [1, 4, 6, 16, 1], 2.5, 5, **kwargs)