Annotation types¶
The user-facing interface exposes a small number of understandable annotation categories, ordered here roughly from cheapest to most expensive for a human to produce.
Classification¶
The contributor assigns a category to an entire image or to an already identified object.
Binary classification: e.g. Animal / No animal.
Multiclass classification: e.g. Mouse / Fox / Deer / Boar.
Multilabel classification: an image may simultaneously carry several labels, e.g. Animal + Water + Vegetation.
Classification may also apply to an existing object, e.g. Root segment #174 → Plant 4.
Detection¶
The contributor draws a bounding box around an object. Typical stored fields: class, x, y, width, height.
Detection answers where are the objects? and how many objects are present? Bounding boxes are less precise than segmentation masks but much faster for humans to create.
Segmentation¶
The contributor identifies the precise pixels belonging to a region or object.
Semantic segmentation: each pixel belongs to a class (e.g. background / root / plate); different instances of the same class touching each other still share one class. U-Net is a typical architecture for this task.
Instance segmentation: individual objects remain distinct (e.g. Root 1, Root 2, Root 3) even when they touch.
Points / landmarks¶
The contributor clicks a specific location, e.g. a root tip, a lateral-root initiation point, an emergence point, a cell center, or an anatomical landmark. Represented simply as class + x + y.
On phones, a click-first mode can offer Tap animal, No animal, and Unsure. It is easier than drawing a precise box and can later seed an automatic box or mask proposal for human confirmation. Multiple points and mobile edit gestures remain later design decisions.
Tracking¶
The contributor follows the same object through successive frames: animal movement, cell movement or division, root growth, developmental time series. Tracking is a later feature: it introduces substantially more interface and data complexity than the types above.
Annotation hierarchy and derived annotations¶
A rich annotation can automatically generate simpler representations:
If a segmentation mask identifies an object, AnnoTrain can automatically calculate its bounding box, centroid, area, and potentially additional morphology measurements, so a contributor never has to annotate the same object twice for different downstream tasks:
The original human annotation remains the source of truth; derived representations are recalculated whenever it changes. This lets a single segmentation dataset support both segmentation and object-detection workflows, but Detection and Segmentation remain separate choices in the interface, because annotation cost matters: a bounding box may take seconds, precise segmentation can take much longer.
General principle:
Ask humans for the least expensive annotation that contains enough information to answer the scientific question.
Guiding project creation without ML jargon¶
Instead of asking researchers "YOLO detection or semantic segmentation?", the interface asks:
What information do you need?
- Is something present?
- What is it?
- Where is it?
- What is its precise shape?
- Where is a specific point?
- How does it move?
AnnoTrain then recommends the appropriate annotation type, e.g. answering "what is its precise shape?" recommends Segmentation, with object detection, bounding box, centroid, and area derived automatically. This is both more accessible and more pedagogical than requiring ML vocabulary up front.
Human annotation is not automatically ground truth¶
AnnoTrain distinguishes a raw human click from a validated annotation. A simple initial consensus strategy shows each image to three independent contributors:
| Annotations | Result |
|---|---|
| Animal, Animal, Animal | Strong consensus |
| Animal, Animal, No animal | Animal, agreement = 0.67 |
Disputed images can receive additional annotations or expert review. Later quality-control mechanisms may include a configurable number of annotations per image, gold-standard images, annotator agreement, expert validation, confidence scores, disagreement detection, and contributor reliability estimates. The raw annotation history always remains available rather than being replaced by the final consensus label.
Redundancy should eventually be adaptive: clear agreement can complete an item after the configured minimum, while ambiguous cases receive another independent annotation. Spatial agreement uses a suitable geometry metric such as overlap.
Human-in-the-loop deep learning¶
A major long-term feature is the feedback loop between humans and models:
Once a model exists, it can generate preliminary annotations that humans Accept / Correct / Reject. The model should eventually prioritize uncertain or informative examples: for binary classification, an image predicted at P(Animal) = 0.998 is less useful to send to a human than one at P(Animal) = 0.51. This is an active-learning workflow. Retraining does not happen after every single annotation; training happens in explicit, versioned cycles once enough new validated annotations exist.
Before a reliable model exists, selection should favor diverse acquisition conditions while retaining some random sampling. Later queues can combine diversity, model uncertainty, human disagreement, quality-control samples, and near-duplicate filtering.
Object selection¶
Selection is for correcting what a program already found: the objects are outlined on the photo, and people say what each one is, which group it belongs to, or both. Nothing in it depends on what the objects are: root segments, cells and animals all work the same way.
A project chooses Object selection in its settings, then gives:
- what an object can be (answers with the "object" role, e.g. Primary, Secondary, Tertiary), and/or
- what objects are grouped into, numbered on each photo (e.g. Plant, Colony, Individual).
The objects come from a file next to each image, <image name>.objects.json, read when the folder
is imported. Importing again updates the objects of photos nobody answered. A photo with answers
keeps its objects, and the import says so: validating said the suggestions shown were right, and the
same id can mean another object in a new file.
Coordinates are in the image's pixels:
{"objects": [
{"id": "a", "line": [[120, 40], [130, 95], [128, 160]]},
{"id": "b", "polygon": [[10, 10], [50, 10], [50, 50]], "label": "secondary", "group": 2},
{"id": "c", "box": [200, 80, 60, 40], "review": true}
]}
label and group are suggestions to correct, and review asks people to look at that object
first. All three are optional. Only photos with objects are served.
Clicking an object toggles it, and dragging takes everything the pointer passes over. A button or a key then answers for all of them: the digits for groups, each answer's letter for answers. Same as... (C) then a click on another object gives them that object's answer, with no button to find among dozens. Each group shows its number at its top on the photo. An answer lists only the objects that person changed, so validating with no change says everything is right. The next person starts from the answers so far.
Export objects in the admin, or python -m app.cli export-objects <project> <file.csv>, gives
each object as it stands now, for a pipeline to read back.