BadBob / Playground
OpenCV · ONNX · MIT
03.01

Prepare your project

Install CMake and OpenCV 4.x. Build the source and run its tests.

Basics

cmake
Configures dependencies and compiles the project.
ctest
Runs automated checks without opening the camera.
Terminalbash
unzip badbob-playground-cpp.zip
cd badbob-playground-cpp
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release --parallel
ctest --test-dir build -C Release --output-on-failure
03.02

Connect your model

Validate the model and labels before opening a camera.

Basics

read_labels(path)
Reads class names in model order; rejects empty or duplicate labels.
OpenCVDetector::detect(image)
Prepares the image, runs the model and returns detections.
Keep in mind
Symptom
Boxes drift or the model fails to load.
Cause
Preprocessing, output layout or label order does not match.
Remedy
This adapter supports only the contract below. Core ML files and other ONNX layouts need a different adapter.

This adapter supports only the contract below. Core ML files and other ONNX layouts need a different adapter.

In depth

Coordinates describe box centers and dimensions in the 640×640 input pixels. Labels follow the exact model class order.

ONNXyaml
input:
  color: RGB
  dtype: float32
  shape: [1, 3, 640, 640]
  letterbox: 114
  scale: 1/255
output:
  dtype: float32
  shape: [1, 4+C, N]
  boxes: [cx, cy, width, height]
  class_scores: probabilities
  objectness: false
  embedded_nms: false
opencv_detector.cppcpp
#include "badbob/opencv_detector.hpp"
#include <fstream>
#include <opencv2/imgproc.hpp>
#include <set>
#include <stdexcept>

namespace badbob {

std::vector<std::string> read_labels(const std::filesystem::path &path) {
    std::ifstream input(path);

    if (!input) {
        throw std::invalid_argument("Cannot open labels file");
    }

    std::vector<std::string> labels;
    std::set<std::string> unique;
    std::string line;

    while (std::getline(input, line)) {
        const auto first = line.find_first_not_of(" \t\r"), last = line.find_last_not_of(" \t\r");

        if (first == std::string::npos) {
            throw std::invalid_argument("Labels must have no empty lines");
        }

        line = line.substr(first, last - first + 1);

        if (line.size() > 120 || line.find_first_of("\n\t") != std::string::npos ||
            !unique.insert(line).second) {
            throw std::invalid_argument(
                "Labels must be unique, single-line names of at most 120 bytes");
        }

        labels.push_back(line);

        if (labels.size() > 1024) {
            throw std::invalid_argument("At most 1024 classes are supported");
        }
    }

    if (input.bad() || labels.empty()) {
        throw std::invalid_argument("Cannot read nonempty labels file");
    }

    return labels;
}

OpenCVDetector::OpenCVDetector(const std::filesystem::path &model, std::size_t classes)
    : classes_(classes) {
    if (model.extension() != ".onnx" || !std::filesystem::is_regular_file(model) || classes == 0 ||
        classes > 1024) {
        throw std::invalid_argument("Provide an existing .onnx detector and 1–1024 labels");
    }

    network_ = cv::dnn::readNetFromONNX(model.string());
    network_.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV);
    network_.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);
    outputs_ = network_.getUnconnectedOutLayersNames();

    if (outputs_.size() != 1) {
        throw std::invalid_argument("Expected exactly one detector output");
    }

    // Fail before camera acquisition if the model cannot consume this exact input
    // contract.
    (void)detect(cv::Mat(640, 640, CV_8UC3, cv::Scalar(114, 114, 114)));
}

std::vector<Detection> OpenCVDetector::detect(const cv::Mat &image) {
    if (image.empty() || image.type() != CV_8UC3) {
        throw std::invalid_argument("Expected a nonempty 8-bit BGR image");
    }

    const auto t = letterbox({image.cols, image.rows});
    cv::Mat resized;
    cv::resize(image, resized, {t.resized.width, t.resized.height});
    cv::Mat padded(t.edge, t.edge, CV_8UC3, cv::Scalar(114, 114, 114));
    resized.copyTo(padded(cv::Rect(t.left, t.top, t.resized.width, t.resized.height)));
    // RGB, NCHW float32, no crop. These values are part of the model contract.
    const auto blob =
        cv::dnn::blobFromImage(padded, 1.0 / 255.0, {640, 640}, {}, true, false, CV_32F);
    network_.setInput(blob);
    cv::Mat output = network_.forward(outputs_.front());

    if (output.dims != 3 || output.type() != CV_32F || output.size[0] != 1 ||
        output.size[1] != static_cast<int>(4 + classes_) || output.size[2] <= 0) {
        throw std::invalid_argument("Expected float32[1,4+C,N], XYWH plus class "
                                    "probabilities; no objectness or embedded NMS");
    }

    if (!output.isContinuous()) {
        output = output.clone();
    }

    return decode({output.ptr<float>(), output.total()}, classes_,
                  static_cast<std::size_t>(output.size[2]), t);
}

} // namespace badbob
03.03

Bring in the camera

Capture and inference share one serial worker. The UI receives only the latest completed image and its detections.

Basics

Runner::run(...)
Handles capture and inference on one worker thread.
Runner::take()
Takes the latest completed image and its detections; results cannot build up in a queue.
Keep in mind
Symptom
The preview lags and memory use grows.
Cause
The camera produces frames faster than inference handles them, so an unbounded queue builds up.
Remedy
Capture and inference share one serial worker. The UI receives only the latest completed image and its detections.
In depth

The ~Runner() destructor requests a stop and waits for the worker. An active OpenCV call must return before cleanup can finish.

runner.cppcpp
#include "badbob/runner.hpp"
#include <exception>
#include <stdexcept>
#include <utility>

namespace badbob {

Runner::Runner(SourceFactory source, DetectorFactory detector)
    : worker_(&Runner::run, this, std::move(source), std::move(detector)) {}

Runner::~Runner() {
    stopped_.store(true);

    // A backend read/forward may still be blocking. Never detach a live owner.
    if (worker_.joinable()) {
        worker_.join();
    }
}

void Runner::run(SourceFactory source, DetectorFactory detector) noexcept {
    try {
        auto model = detector();

        if (!model) {
            throw std::runtime_error("Missing detector adapter");
        }

        if (stopped_.load()) {
            return;
        }

        auto frames = source();

        if (!frames) {
            throw std::runtime_error("Missing capture adapter");
        }

        while (!stopped_.load()) {
            auto image = frames->next();

            if (stopped_.load()) {
                break;
            }

            if (image.empty()) {
                throw std::runtime_error("Camera returned an empty frame");
            }

            auto detections = model->detect(image);

            if (stopped_.load()) {
                break;
            }

            result_.publish({std::move(image), std::move(detections), {}});
        }
    } catch (const std::exception &error) {
        if (!stopped_.load()) {
            result_.publish({{}, {}, error.what()});
        }
    } catch (...) {
        if (!stopped_.load()) {
            result_.publish({{}, {}, "Unknown capture/inference failure"});
        }
    }
}

} // namespace badbob
03.04

Put predictions in place

Undo letterbox scaling and padding, then clip boxes before drawing.

Basics

letterbox(...) · unproject(...)
Preserves the aspect ratio and maps boxes back to the original image.
decode(...)
Validates predictions and removes overlapping boxes of the same class.
Keep in mind

Treat image size, crop, orientation and coordinates as one contract. Test it with known boxes before trusting an overlay.

detection.cppcpp
#include "badbob/detection.hpp"
#include <algorithm>
#include <cmath>
#include <limits>
#include <stdexcept>

namespace badbob {
namespace {

void validate(const Letterbox &t) {
    if (t.original.width <= 0 || t.original.height <= 0 || t.resized.width <= 0 ||
        t.resized.height <= 0 || t.edge <= 0 || t.resized.width > t.edge ||
        t.resized.height > t.edge || t.left < 0 || t.top < 0 || t.left > t.edge - t.resized.width ||
        t.top > t.edge - t.resized.height) {
        throw std::invalid_argument("Invalid letterbox geometry");
    }
}

double intersection_over_union(Box a, Box b) {
    const double w = std::max(0.0, std::min(a.x + a.width, b.x + b.width) - std::max(a.x, b.x));
    const double h = std::max(0.0, std::min(a.y + a.height, b.y + b.height) - std::max(a.y, b.y));
    const double intersection = w * h;
    const double area = a.width * a.height + b.width * b.height - intersection;

    return area > 0 ? intersection / area : 0;
}

} // namespace

Letterbox letterbox(Size size, int edge) {
    if (size.width <= 0 || size.height <= 0 || edge <= 0 || edge > 4096) {
        throw std::invalid_argument(
            "Image and input dimensions must be positive; input edge <=4096");
    }

    const double ratio = std::min(double(edge) / size.width, double(edge) / size.height);
    const Size resized{std::max(1, int(std::lround(size.width * ratio))),
                       std::max(1, int(std::lround(size.height * ratio)))};

    return {size, resized, (edge - resized.width) / 2, (edge - resized.height) / 2, edge};
}

Box unproject(Box b, const Letterbox &t) {
    validate(t);

    if (!std::isfinite(b.x) || !std::isfinite(b.y) || !std::isfinite(b.width) ||
        !std::isfinite(b.height) || !std::isfinite(b.x + b.width) ||
        !std::isfinite(b.y + b.height) || b.width < 0 || b.height < 0) {
        throw std::invalid_argument("Box coordinates must be finite with nonnegative dimensions");
    }

    const auto x = [&](double v) {
        return std::clamp((v - t.left) / t.resized.width, 0.0, 1.0) * t.original.width;
    };
    const auto y = [&](double v) {
        return std::clamp((v - t.top) / t.resized.height, 0.0, 1.0) * t.original.height;
    };
    const double left = x(b.x), top = y(b.y);

    return {left, top, x(b.x + b.width) - left, y(b.y + b.height) - top};
}

std::vector<Detection> decode(std::span<const float> data, std::size_t classes, std::size_t count,
                              const Letterbox &t, float confidence, float overlap) {
    validate(t);

    if (classes == 0 || classes > 1024 || count == 0 || count > 100000 ||
        data.size() != (4 + classes) * count || !std::isfinite(confidence) ||
        !std::isfinite(overlap) || confidence < 0 || confidence > 1 || overlap < 0 || overlap > 1) {
        throw std::invalid_argument("Expected [1,4+C,N] probabilities, C<=1024, "
                                    "N<=100000 and thresholds in [0,1]");
    }

    std::vector<Detection> candidates;

    for (std::size_t i = 0; i < count; ++i) {
        const double cx = data[i], cy = data[count + i], w = data[2 * count + i],
                     h = data[3 * count + i];

        if (!std::isfinite(cx) || !std::isfinite(cy) || !std::isfinite(w) || !std::isfinite(h) ||
            w < 0 || h < 0) {
            throw std::invalid_argument("Unsupported detector coordinates");
        }

        float score = 0;
        std::size_t label = 0;

        for (std::size_t c = 0; c < classes; ++c) {
            const float probability = data[(4 + c) * count + i];

            if (!std::isfinite(probability) || probability < 0 || probability > 1) {
                throw std::invalid_argument(
                    "Expected probabilities in [0,1], not logits/objectness");
            }

            if (probability > score) {
                score = probability;
                label = c;
            }
        }

        if (score < confidence) {
            continue;
        }

        const auto box = unproject({cx - w / 2, cy - h / 2, w, h}, t);

        if (box.width > 0 && box.height > 0) {
            candidates.push_back({box, label, score});
        }
    }

    std::stable_sort(candidates.begin(), candidates.end(), [](const auto &a, const auto &b) {
        return a.score > b.score;
    });
    // Bound quadratic suppression work; keep the highest-scoring candidates
    // first.
    if (candidates.size() > 3000) {
        candidates.resize(3000);
    }

    std::vector<Detection> selected;

    for (const auto &candidate : candidates) {
        const bool suppressed =
            std::any_of(selected.begin(), selected.end(), [&](const auto &previous) {
                return previous.label == candidate.label &&
                       intersection_over_union(previous.box, candidate.box) > overlap;
            });

        if (!suppressed) {
            selected.push_back(candidate);
        }

        if (selected.size() == 300) {
            break;
        }
    }

    return selected;
}

} // namespace badbob
03.05

Build. Run. Try your model.

Select a camera explicitly, or process an image offline. Press Q or Escape to stop.

Basics

parse(argc, argv)
Validates arguments and selects exactly one input: image or camera.
run_image(...) · run_camera(...)
Runs the chosen path; camera mode delegates processing to Runner.
Keep in mind
Symptom
Stopping takes longer than expected.
Cause
An OpenCV capture or inference call is still blocking.
Remedy
Request stop, reject new results and join the worker. Backend timeouts require platform support; never detach live resources.
In depth

write_new_png() creates a new file exclusively; it never overwrites an existing file.

Terminalbash
./build/badbob-camera --model detector.onnx --labels labels.txt --image frame.png --output result.png
./build/badbob-camera --model detector.onnx --labels labels.txt --camera 0

03.06 / Best practices

Small details. Better examples.

  • Keep capture and inference off the UI thread. Bound the queue instead of collecting a backlog of old frames.
  • Treat image size, crop, orientation and coordinates as one contract. Test it with known boxes before trusting an overlay.
  • Request stop, reject new results and join the worker. Backend timeouts require platform support; never detach live resources.