03 / C++
C++. From model to camera.
Run your ONNX detector locally with live annotations and bounded frame processing.
These examples focus on objects and animals in controlled settings. They do not cover people surveillance, identification or behavioral analysis.
Prepare your project
Install CMake and OpenCV 4.x. Build the source and run its tests.
Basics
cmake- Configures dependencies and compiles the project.
ctest- Runs automated checks without opening the camera.
unzip badbob-playground-cpp.zip
cd badbob-playground-cpp
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release --parallel
ctest --test-dir build -C Release --output-on-failureConnect your model
Validate the model and labels before opening a camera.
Basics
read_labels(path)- Reads class names in model order; rejects empty or duplicate labels.
OpenCVDetector::detect(image)- Prepares the image, runs the model and returns detections.
Keep in mind
- Symptom
- Boxes drift or the model fails to load.
- Cause
- Preprocessing, output layout or label order does not match.
- Remedy
- This adapter supports only the contract below. Core ML files and other ONNX layouts need a different adapter.
This adapter supports only the contract below. Core ML files and other ONNX layouts need a different adapter.
In depth
Coordinates describe box centers and dimensions in the 640×640 input pixels. Labels follow the exact model class order.
input:
color: RGB
dtype: float32
shape: [1, 3, 640, 640]
letterbox: 114
scale: 1/255
output:
dtype: float32
shape: [1, 4+C, N]
boxes: [cx, cy, width, height]
class_scores: probabilities
objectness: false
embedded_nms: false#include "badbob/opencv_detector.hpp"
#include <fstream>
#include <opencv2/imgproc.hpp>
#include <set>
#include <stdexcept>
namespace badbob {
std::vector<std::string> read_labels(const std::filesystem::path &path) {
std::ifstream input(path);
if (!input) {
throw std::invalid_argument("Cannot open labels file");
}
std::vector<std::string> labels;
std::set<std::string> unique;
std::string line;
while (std::getline(input, line)) {
const auto first = line.find_first_not_of(" \t\r"), last = line.find_last_not_of(" \t\r");
if (first == std::string::npos) {
throw std::invalid_argument("Labels must have no empty lines");
}
line = line.substr(first, last - first + 1);
if (line.size() > 120 || line.find_first_of("\n\t") != std::string::npos ||
!unique.insert(line).second) {
throw std::invalid_argument(
"Labels must be unique, single-line names of at most 120 bytes");
}
labels.push_back(line);
if (labels.size() > 1024) {
throw std::invalid_argument("At most 1024 classes are supported");
}
}
if (input.bad() || labels.empty()) {
throw std::invalid_argument("Cannot read nonempty labels file");
}
return labels;
}
OpenCVDetector::OpenCVDetector(const std::filesystem::path &model, std::size_t classes)
: classes_(classes) {
if (model.extension() != ".onnx" || !std::filesystem::is_regular_file(model) || classes == 0 ||
classes > 1024) {
throw std::invalid_argument("Provide an existing .onnx detector and 1–1024 labels");
}
network_ = cv::dnn::readNetFromONNX(model.string());
network_.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV);
network_.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);
outputs_ = network_.getUnconnectedOutLayersNames();
if (outputs_.size() != 1) {
throw std::invalid_argument("Expected exactly one detector output");
}
// Fail before camera acquisition if the model cannot consume this exact input
// contract.
(void)detect(cv::Mat(640, 640, CV_8UC3, cv::Scalar(114, 114, 114)));
}
std::vector<Detection> OpenCVDetector::detect(const cv::Mat &image) {
if (image.empty() || image.type() != CV_8UC3) {
throw std::invalid_argument("Expected a nonempty 8-bit BGR image");
}
const auto t = letterbox({image.cols, image.rows});
cv::Mat resized;
cv::resize(image, resized, {t.resized.width, t.resized.height});
cv::Mat padded(t.edge, t.edge, CV_8UC3, cv::Scalar(114, 114, 114));
resized.copyTo(padded(cv::Rect(t.left, t.top, t.resized.width, t.resized.height)));
// RGB, NCHW float32, no crop. These values are part of the model contract.
const auto blob =
cv::dnn::blobFromImage(padded, 1.0 / 255.0, {640, 640}, {}, true, false, CV_32F);
network_.setInput(blob);
cv::Mat output = network_.forward(outputs_.front());
if (output.dims != 3 || output.type() != CV_32F || output.size[0] != 1 ||
output.size[1] != static_cast<int>(4 + classes_) || output.size[2] <= 0) {
throw std::invalid_argument("Expected float32[1,4+C,N], XYWH plus class "
"probabilities; no objectness or embedded NMS");
}
if (!output.isContinuous()) {
output = output.clone();
}
return decode({output.ptr<float>(), output.total()}, classes_,
static_cast<std::size_t>(output.size[2]), t);
}
} // namespace badbobBring in the camera
Capture and inference share one serial worker. The UI receives only the latest completed image and its detections.
Basics
Runner::run(...)- Handles capture and inference on one worker thread.
Runner::take()- Takes the latest completed image and its detections; results cannot build up in a queue.
Keep in mind
- Symptom
- The preview lags and memory use grows.
- Cause
- The camera produces frames faster than inference handles them, so an unbounded queue builds up.
- Remedy
- Capture and inference share one serial worker. The UI receives only the latest completed image and its detections.
In depth
The ~Runner() destructor requests a stop and waits for the worker. An active OpenCV call must return before cleanup can finish.
#include "badbob/runner.hpp"
#include <exception>
#include <stdexcept>
#include <utility>
namespace badbob {
Runner::Runner(SourceFactory source, DetectorFactory detector)
: worker_(&Runner::run, this, std::move(source), std::move(detector)) {}
Runner::~Runner() {
stopped_.store(true);
// A backend read/forward may still be blocking. Never detach a live owner.
if (worker_.joinable()) {
worker_.join();
}
}
void Runner::run(SourceFactory source, DetectorFactory detector) noexcept {
try {
auto model = detector();
if (!model) {
throw std::runtime_error("Missing detector adapter");
}
if (stopped_.load()) {
return;
}
auto frames = source();
if (!frames) {
throw std::runtime_error("Missing capture adapter");
}
while (!stopped_.load()) {
auto image = frames->next();
if (stopped_.load()) {
break;
}
if (image.empty()) {
throw std::runtime_error("Camera returned an empty frame");
}
auto detections = model->detect(image);
if (stopped_.load()) {
break;
}
result_.publish({std::move(image), std::move(detections), {}});
}
} catch (const std::exception &error) {
if (!stopped_.load()) {
result_.publish({{}, {}, error.what()});
}
} catch (...) {
if (!stopped_.load()) {
result_.publish({{}, {}, "Unknown capture/inference failure"});
}
}
}
} // namespace badbobPut predictions in place
Undo letterbox scaling and padding, then clip boxes before drawing.
Basics
letterbox(...) · unproject(...)- Preserves the aspect ratio and maps boxes back to the original image.
decode(...)- Validates predictions and removes overlapping boxes of the same class.
Keep in mind
Treat image size, crop, orientation and coordinates as one contract. Test it with known boxes before trusting an overlay.
#include "badbob/detection.hpp"
#include <algorithm>
#include <cmath>
#include <limits>
#include <stdexcept>
namespace badbob {
namespace {
void validate(const Letterbox &t) {
if (t.original.width <= 0 || t.original.height <= 0 || t.resized.width <= 0 ||
t.resized.height <= 0 || t.edge <= 0 || t.resized.width > t.edge ||
t.resized.height > t.edge || t.left < 0 || t.top < 0 || t.left > t.edge - t.resized.width ||
t.top > t.edge - t.resized.height) {
throw std::invalid_argument("Invalid letterbox geometry");
}
}
double intersection_over_union(Box a, Box b) {
const double w = std::max(0.0, std::min(a.x + a.width, b.x + b.width) - std::max(a.x, b.x));
const double h = std::max(0.0, std::min(a.y + a.height, b.y + b.height) - std::max(a.y, b.y));
const double intersection = w * h;
const double area = a.width * a.height + b.width * b.height - intersection;
return area > 0 ? intersection / area : 0;
}
} // namespace
Letterbox letterbox(Size size, int edge) {
if (size.width <= 0 || size.height <= 0 || edge <= 0 || edge > 4096) {
throw std::invalid_argument(
"Image and input dimensions must be positive; input edge <=4096");
}
const double ratio = std::min(double(edge) / size.width, double(edge) / size.height);
const Size resized{std::max(1, int(std::lround(size.width * ratio))),
std::max(1, int(std::lround(size.height * ratio)))};
return {size, resized, (edge - resized.width) / 2, (edge - resized.height) / 2, edge};
}
Box unproject(Box b, const Letterbox &t) {
validate(t);
if (!std::isfinite(b.x) || !std::isfinite(b.y) || !std::isfinite(b.width) ||
!std::isfinite(b.height) || !std::isfinite(b.x + b.width) ||
!std::isfinite(b.y + b.height) || b.width < 0 || b.height < 0) {
throw std::invalid_argument("Box coordinates must be finite with nonnegative dimensions");
}
const auto x = [&](double v) {
return std::clamp((v - t.left) / t.resized.width, 0.0, 1.0) * t.original.width;
};
const auto y = [&](double v) {
return std::clamp((v - t.top) / t.resized.height, 0.0, 1.0) * t.original.height;
};
const double left = x(b.x), top = y(b.y);
return {left, top, x(b.x + b.width) - left, y(b.y + b.height) - top};
}
std::vector<Detection> decode(std::span<const float> data, std::size_t classes, std::size_t count,
const Letterbox &t, float confidence, float overlap) {
validate(t);
if (classes == 0 || classes > 1024 || count == 0 || count > 100000 ||
data.size() != (4 + classes) * count || !std::isfinite(confidence) ||
!std::isfinite(overlap) || confidence < 0 || confidence > 1 || overlap < 0 || overlap > 1) {
throw std::invalid_argument("Expected [1,4+C,N] probabilities, C<=1024, "
"N<=100000 and thresholds in [0,1]");
}
std::vector<Detection> candidates;
for (std::size_t i = 0; i < count; ++i) {
const double cx = data[i], cy = data[count + i], w = data[2 * count + i],
h = data[3 * count + i];
if (!std::isfinite(cx) || !std::isfinite(cy) || !std::isfinite(w) || !std::isfinite(h) ||
w < 0 || h < 0) {
throw std::invalid_argument("Unsupported detector coordinates");
}
float score = 0;
std::size_t label = 0;
for (std::size_t c = 0; c < classes; ++c) {
const float probability = data[(4 + c) * count + i];
if (!std::isfinite(probability) || probability < 0 || probability > 1) {
throw std::invalid_argument(
"Expected probabilities in [0,1], not logits/objectness");
}
if (probability > score) {
score = probability;
label = c;
}
}
if (score < confidence) {
continue;
}
const auto box = unproject({cx - w / 2, cy - h / 2, w, h}, t);
if (box.width > 0 && box.height > 0) {
candidates.push_back({box, label, score});
}
}
std::stable_sort(candidates.begin(), candidates.end(), [](const auto &a, const auto &b) {
return a.score > b.score;
});
// Bound quadratic suppression work; keep the highest-scoring candidates
// first.
if (candidates.size() > 3000) {
candidates.resize(3000);
}
std::vector<Detection> selected;
for (const auto &candidate : candidates) {
const bool suppressed =
std::any_of(selected.begin(), selected.end(), [&](const auto &previous) {
return previous.label == candidate.label &&
intersection_over_union(previous.box, candidate.box) > overlap;
});
if (!suppressed) {
selected.push_back(candidate);
}
if (selected.size() == 300) {
break;
}
}
return selected;
}
} // namespace badbobBuild. Run. Try your model.
Select a camera explicitly, or process an image offline. Press Q or Escape to stop.
Basics
parse(argc, argv)- Validates arguments and selects exactly one input: image or camera.
run_image(...) · run_camera(...)- Runs the chosen path; camera mode delegates processing to Runner.
Keep in mind
- Symptom
- Stopping takes longer than expected.
- Cause
- An OpenCV capture or inference call is still blocking.
- Remedy
- Request stop, reject new results and join the worker. Backend timeouts require platform support; never detach live resources.
In depth
write_new_png() creates a new file exclusively; it never overwrites an existing file.
./build/badbob-camera --model detector.onnx --labels labels.txt --image frame.png --output result.png
./build/badbob-camera --model detector.onnx --labels labels.txt --camera 003.06 / Best practices
Small details. Better examples.
- Keep capture and inference off the UI thread. Bound the queue instead of collecting a backlog of old frames.
- Treat image size, crop, orientation and coordinates as one contract. Test it with known boxes before trusting an overlay.
- Request stop, reject new results and join the worker. Backend timeouts require platform support; never detach live resources.