Skip to content

Technical architecture

Note

This describes the target architecture. Only the parts marked as implemented in the Roadmap currently exist in the repository.

Overview

Browser
   │
   ▼
Frontend
   │
   ▼
FastAPI API
   ├────────► PostgreSQL
   │
   ├────────► Image / Object Storage
   │
   └────────► Job Queue
                    │
                    ▼
               ML Worker
                    │
                    ▼
                  GPU

The existing Vermeer Lab website (built with Quarto) remains the portal and documentation layer: it links out to the AnnoTrain application on a dedicated route or subdomain.

AnnoTrain is one responsive multi-user web application for phones, laptops, and desktops. Ordinary HTTP requests are sufficient for the initial annotation loop; WebSockets are not required. The server selects an eligible item, returns a lightweight representation, stores the response, and selects the next item. Short-lived reservations may later reduce accidental duplicate assignment without preventing intentional independent annotations for consensus.

Backend

Initial choice: Python + FastAPI. The API is expected to eventually expose routes conceptually similar to:

GET  /projects
POST /projects

GET  /media/{id}

POST /annotations

POST /training-jobs
GET  /training-jobs/{id}

GET  /models/{id}

Database

Target database: PostgreSQL, storing metadata and annotations, not the original scientific images themselves.

Image storage

Scientific images live in file/object storage, initially potentially on the existing lab NAS. The backend stores references to these files rather than the files themselves. Long term, an S3-compatible abstraction would make the storage layer portable.

Compute

Training and inference run on separate workers with access to suitable CPU/GPU resources. The public API never directly exposes the GPU server.

Image handling

Original scientific data remains unchanged. Browser display and model preprocessing are separate derivatives:

Raw original
├── Web derivative: lightweight display and annotation
└── Model input: controlled, reproducible preprocessing

Browser compression never implicitly defines model input. Model preparation may resize, tile, normalize, or crop data reproducibly. Spatial annotation coordinates must always map back to the original.

The web version may be roughly 1600 pixels wide and compressed for interactive annotation; originals remain available for training, export, and high-resolution analysis.

Initial video support extracts representative timestamped frames rather than implementing tracking:

Raw video → frame every N seconds → optional near-duplicate removal → annotate

Storage principles

Git must never be used for scientific datasets or trained model weights. The repository contains source code, documentation, configuration templates, tests, and lightweight examples only. Large data stays in external storage.

Potential storage organization:

projects/
└── <project-id>/
    ├── original/
    ├── web/
    ├── annotations/
    ├── datasets/
    │   ├── v001/
    │   └── v002/
    └── models/
        ├── v001/
        └── v002/

Media should eventually be reusable across projects. Content hashes can identify identical files, while project permissions still control access. Project merging is not an MVP priority; traceability and interoperability are.

Annotation engine vs. ML engine

AnnoTrain is conceptually split into two systems, so a researcher can use it purely as an annotation/dataset platform without enabling deep learning.

Annotation engine: projects, media, contributors, tasks, classes, annotations, consensus, quality control, dataset creation.

ML engine: training jobs, inference jobs, evaluation, model versions, predictions, uncertainty, model publication. The dataset-export → preprocessing → training → evaluation pipeline is planned to be built with Snakemake for reproducibility, kept entirely on the ML-engine side - it does not replace or change the FastAPI web API used by the annotation engine.

An optional future project setting might be "automatically train a baseline model".

Open science and model sharing

A trained model is more than a .pt file. Each ModelVersion should eventually contain or reference: model architecture, weights, task type, input format/dimensions, preprocessing, output classes/format, training configuration, dataset version, evaluation metrics, date, software/code version, license, limitations, and a model card.

Possible model actions: Try model, Use for annotation, Download model, Fine-tune, Publish.

Potential future integrations (not required for the MVP): GitHub for source code, Hugging Face for model/dataset distribution, Zenodo for archival releases and citations.