Start by counting what happened
A real object matched by an accepted detection is a true positive, TP. An accepted detection without a correct match is a false positive, FP. A real object left unmatched is a false negative, FN.
For objects and masks, first define a match: the right class, enough overlap and a clear rule preventing duplicate matches. Otherwise counts from different runs mean different things.
Go deeper
Two predictions pointing at the same real object cannot both count as separate correct detections. The examples below start after matching, so you can focus on the threshold. They do not run NMS or mask matching in the browser.
How many detections are right — and how many objects are found?
Precision asks: of the detections shown, how many were right? Recall asks: of the objects actually present, how many were found?
Raise or lower the threshold in the example. Fewer detections do not automatically mean a better model. Choose a balance based on which errors matter for your task.
- Precision
- 75%
- Recall
- 60%
- Correctly found
- 3
- False detections
- 1
- Missed
- 2
| Candidate | Score | Judged match | Shown |
|---|---|---|---|
| A | 0.96 | Correct | Shown |
| X | 0.88 | Incorrect | Shown |
| B | 0.76 | Correct | Shown |
| C | 0.61 | Correct | Shown |
| Y | 0.42 | Incorrect | Hidden |
| D | 0.31 | Correct | Hidden |
Go deeper
Precision = TP / (TP + FP). Recall = TP / (TP + FN). When no detections are shown, precision is undefined in this example; it should not automatically be called 100%.
With five real objects and threshold 0.50, TP=3, FP=1 and FN=2: precision 75% and recall 60%. Candidate match judgements are fixed. This is a toy table, not a BadBob quality measurement.
Finding an object is different from following its boundary
In BadBob’s experimental recipe · Recipe reviewed 29 October 2026
Intersection over Union, IoU, divides the intersection of two regions by their union. For masks it counts pixels; for boxes it uses rectangle areas.
Two masks sharing 60 pixels and covering 100 pixels in total have IoU 0.60. A box may have high IoU while the mask still loses a narrow part. Examine the examples as well as averages.
Go deeper
BadBob’s experimental evaluation reports TP, FP, FN and meanBestMaskIoU, among other values. The latter averages each ground-truth instance’s best same-class mask overlap. It is not a replacement for one-to-one matching or an AP/mAP protocol.
An average can hide a weak class or kind of boundary. Break results down by class, size and relevant conditions. AP summarizes a precision–recall curve; its IoU and averaging protocol must be specified.
A high score needs testing on new images
A model can fit training material very well and still fail in new situations. Lighting, background, distance, object size and occlusion are all part of the task.
Failure Bank helps retain concrete mistakes and decide their future role. Protected regression examples should not move into training just to make an evaluation number look better.
Go deeper
Define your comparison protocol before comparing runs: the same data version, class mapping, thresholds and matching rules. Improve the weak part of the system — data, annotations, recipe or postprocessing — then assess again.