Skip to content

Roadmap

The roadmap stays flexible and is driven by actual scientific users. A strong signal for investing more development time is: another research group independently wants to use AnnoTrain.

Current status

Implemented (V0 → V0.4, awaiting a real-world rehearsal):

  • [x] Project foundation, GitHub repository and this documentation site
  • [x] FastAPI backend with a generic data model (projects, tasks, labels, media, contributors, append-only annotations)
  • [x] Image import with SHA-256 de-duplication and separate browser derivatives
  • [x] Anonymous contributors, session codes, least-annotated-first assignment, undo
  • [x] Consensus (plurality vote, agreement, disputed items) and CSV exports
  • [x] React annotation screen (mobile-first, keyboard shortcuts, prefetching, offline-tolerant sync) and a live dashboard for projection
  • [x] Docker Compose stack (PostgreSQL, API, nginx) with optional Cloudflare Tunnel, backups and a load-test script

Next:

  • [ ] Deploy on the remote machine and rehearse with a few colleagues (see Deployment)
  • [ ] Import the Caltech Camera Traps subset and prepare a fallback dataset/model
  • [ ] Run the load test against the deployed stack
  • [ ] Baseline training pipeline driven by Snakemake (V2/V3)

Version progression

V0: Project foundation. Clean, documented, version-controlled project.

V0.1: Minimal backend. Python + FastAPI + Uvicorn, first endpoint GET /health.

V0.2: Minimal annotation workflow. Load an image, choose Animal / No animal / Unsure, save annotation, show next image.

V0.3: Persistent database. Introduce PostgreSQL to store projects, media, contributors, and annotations.

V0.4: Classroom-ready prototype. Support ~40 simultaneous students: anonymous contributor IDs, project/session links, annotation progress, basic consensus, web thumbnails, safe concurrent item assignment, robust deployment.

V1: Generic classification projects. Researchers create their own classification projects without touching source code: project creation, image import, optional timestamped video-frame extraction, class configuration, binary/multiclass tasks, public/unlisted projects, exports.

V2: Dataset quality and versioning. Multiple annotations per image, adaptive redundancy, consensus, disagreement review, DatasetVersion, train/validation/test splitting.

V3: Deep-learning training. Training jobs, pretrained image backbones, evaluation, confusion matrices, ModelVersion, prediction visualization.

V4: Human-in-the-loop. Model pre-annotations, Accept / Correct / Reject, uncertainty, diversity and near-duplicate-aware sampling, active-learning queues.

V5: Spatial annotations. Progressively: click-first points, bounding boxes, segmentation, lines/paths, and optional task pipelines, with derived annotations generated automatically where possible.

V6: Open model/dataset sharing. Model cards, reproducible training configurations, dataset/model downloads, external publishing integrations.

V7: Tracking/video. Full temporal tracking only after simpler workflows are mature; basic video-to-frame ingestion can arrive earlier.

Milestones

November 2026: a classroom-ready participatory binary classification workflow works reliably (see use cases).

End of 2026: another researcher can create a classification project without source-code modifications.

2027: progressively add spatial annotations, segmentation, existing-model inference, automated training, model sharing, and active learning.

Initial dataset candidate

Camera-trap images are a good first pedagogical dataset because the task is intuitive. A useful candidate is the Caltech Camera Traps dataset distributed through LILA BC. For development, AnnoTrain uses only a small subset rather than an entire large dataset, even 50–100 local images are enough for a first technical prototype.

Planned ML pipeline (V3/V4, not yet built)

Once training jobs exist, the plan is to keep architectures simple and well understood rather than exotic: ResNet-18 for classification, U-Net for semantic segmentation (and for point/landmark prediction via heatmap regression), Mask R-CNN for instance segmentation, and CenterNet as a detector able to learn from a mix of point and bounding-box annotations. A model-assisted loop (prediction → human Accept/Correct/Reject → new DatasetVersion → new ModelVersion) and active learning (prioritizing low-confidence or disputed images for review) follow the same versioned-retraining principle as the rest of the roadmap: retrain in explicit batches, never after every single annotation.

The dataset-preparation and training pipeline itself (export → preprocessing → training → evaluation) is planned to be built with Snakemake, so each step stays declarative, reproducible, and resumable; it belongs to the ML engine and does not change the FastAPI web API. Longer term, AnnoTrain aims to support reusing existing external models (starting with an internal model package plus plain PyTorch import) - MegaDetector is a strong first candidate for camera-trap projects. When reusing a model across projects (e.g. a different park's cameras), it should always be benchmarked against a fine-tuned or from-scratch alternative rather than assumed to transfer, since camera-trap datasets are prone to domain shift and to train/test leakage from near-identical consecutive frames (split by camera or recording event, not randomly by image).