From layers to an individual mask

BADBOB / IN DEPTH · 14 min

Inside BadBob’s experimental YOLACT architecture.

Reading draft · Svenska and English

We review the writing and illustrations in Swedish and English before adding more languages.

Every object gets its own shape

In BadBob’s experimental recipe · Recipe reviewed 29 October 2026

An instance mask asks which pixels belong to this particular object. Two touching apples need two separate masks.

BadBob’s experimental segmentation uses YOLACT with a ResNet-50 backbone. This differs from its Create ML detector, which can use Darknet YOLO or transfer learning with ObjectPrint. Different model types do not all share one layer architecture.

AppleAppleCup
Apple
Cup

Which objects are present, and where? A class and a box for each detection.

Teaching illustration — not a model run or measured activations.
Go deeper

Contours are editable geometric boundaries. Masks represent pixel regions. Converting between them can affect small holes, narrow details and edges. Review both annotation geometry and the mask actually used for training.

ResNet-50, part by part

The backbone builds image features. ResNet uses residual blocks, with a shortcut that adds information to the block’s transformation.

The configured ResNet-50 backbone has four groups of 3, 4, 6 and 3 bottleneck blocks. Each bottleneck combines 1×1, 3×3 and 1×1 convolutions. The number of groups is different from the layer count in the model name.

Image + preprocessing

RGB pixels are resized to the recipe square and normalized per channel. This is not a display of actual activations.

Stem

The initial convolution and pooling reduce spatial resolution and build the first features.

C2 · 3 blocks

The first residual group transforms the representation. Many channels carry detail and context.

C3 · 4 blocks

This group supplies one of the feature-pyramid inputs in the configured network.

C4 · 6 blocks

More residual blocks process features across larger receptive fields. This group also feeds the pyramid.

C5 · 3 blocks

The deepest of these groups supplies more context to the pyramid. Smaller spatial maps do not mean fewer features.

Teaching illustration — not a model run or measured activations.
Go deeper

A residual connection can be written schematically as y = F(x) + x. If dimensions differ, the shortcut may need a projection. A bottleneck reduces channel width, processes local information and then expands the channels.

The illustration shows block groups and flow, not every operation, exact tensor dimensions or measured filter responses. Actual shapes depend on input size and the architecture’s stride and padding.

Large and small, together

A feature pyramid combines representations at different scales. Prediction heads can use both spatial information and deeper context.

In the selected YOLACT configuration, C3, C4 and C5 feed the pyramid. Its features supply predictions and the prototype-mask branch.

Go deeper

FPN uses a top-down pathway with lateral connections to combine scales. Multiple pyramid levels let the network look for objects at different sizes. This does not guarantee small-object detection: detail already lost from the input cannot be conjured back.

Two branches build a mask

One branch produces shared prototype masks for the image. The other predicts boxes, class scores and mask coefficients for candidate detections.

For a selected object, its coefficients combine the prototypes. The result is converted to mask values, cropped by the box and thresholded into a pixel region. Prototypes are not a catalogue of finished object classes.

  1. RGB image
  2. ResNet-50
  3. Feature pyramid
Shared prototype masks
Boxes · classes · coefficients

Combine → crop → mask

Two branches share image features. Prototypes combine with each object’s own coefficients.

Teaching illustration — not a model run or measured activations.
Go deeper

The pinned prototype branch ends in 32 channels. An object’s 32 coefficients form a weighted combination of these prototypes. Schematically: Mᵢ = sigmoid(P · cᵢ). BadBob inference crops the result, interpolates it to recipe resolution and compares it with 0.5.

Multiple objects can use the same prototype with different coefficients. This shares image computation while constructing distinct instance masks. None of the illustration’s coloured plates is an actual prototype from a run.

What BadBob actually runs

In BadBob’s experimental recipe · Recipe reviewed 29 October 2026

The current experimental recipe runs locally on CPU and permits image sizes 128, 256 or 512. It stretches images to a square; this path does not preserve aspect ratio through letterboxing.

Inference selects candidates by class score, applies box NMS to remove overlapping duplicates, then builds masks for the remaining detections. GPU speed measurements from the paper do not describe this CPU recipe.

Go deeper

The recipe uses SGD. Its inference protocol uses class scores at least 0.25, box-NMS IoU 0.5, at most 40 candidates per class before selection, and at most 10 final detections. Evaluation matches masks class-wise and one-to-one at mask IoU at least 0.5.

Preprocessing subtracts RGB means 123.68 / 116.78 / 103.94 and divides channels by 58.40 / 57.12 / 57.38. Model, normalization and class order belong together. The explanation needs updating when the recipe changes.

Sources and further reading