Technical architecture¶
Note
This describes the target architecture. Only the parts marked as implemented in the Roadmap currently exist in the repository.
Overview¶
Browser
│
▼
Frontend
│
▼
FastAPI API
├────────► PostgreSQL
│
├────────► Image / Object Storage
│
└────────► Job Queue
│
▼
ML Worker
│
▼
GPU
The existing Vermeer Lab website (built with Quarto) remains the portal and documentation layer: it links out to the AnnoTrain application on a dedicated route or subdomain.
AnnoTrain is one responsive multi-user web application for phones, laptops, and desktops. Ordinary HTTP requests are sufficient for the initial annotation loop; WebSockets are not required. The server selects an eligible item, returns a lightweight representation, stores the response, and selects the next item. Short-lived reservations may later reduce accidental duplicate assignment without preventing intentional independent annotations for consensus.
Backend¶
Initial choice: Python + FastAPI. The API is expected to eventually expose routes conceptually similar to:
GET /projects
POST /projects
GET /media/{id}
POST /annotations
POST /training-jobs
GET /training-jobs/{id}
GET /models/{id}
Database¶
Target database: PostgreSQL, storing metadata and annotations, not the original scientific images themselves.
Image storage¶
Scientific images live in file/object storage, initially potentially on the existing lab NAS. The backend stores references to these files rather than the files themselves. Long term, an S3-compatible abstraction would make the storage layer portable.
Compute¶
Training and inference run on separate workers with access to suitable CPU/GPU resources. The public API never directly exposes the GPU server.
Image handling¶
Original scientific data remains unchanged. Browser display and model preprocessing are separate derivatives:
Raw original
├── Web derivative: lightweight display and annotation
└── Model input: controlled, reproducible preprocessing
Browser compression never implicitly defines model input. Model preparation may resize, tile, normalize, or crop data reproducibly. Spatial annotation coordinates must always map back to the original.
The web version may be roughly 1600 pixels wide and compressed for interactive annotation; originals remain available for training, export, and high-resolution analysis.
Initial video support extracts representative timestamped frames rather than implementing tracking:
Storage principles¶
Git must never be used for scientific datasets or trained model weights. The repository contains source code, documentation, configuration templates, tests, and lightweight examples only. Large data stays in external storage.
Potential storage organization:
projects/
└── <project-id>/
├── original/
├── web/
├── annotations/
├── datasets/
│ ├── v001/
│ └── v002/
└── models/
├── v001/
└── v002/
Media should eventually be reusable across projects. Content hashes can identify identical files, while project permissions still control access. Project merging is not an MVP priority; traceability and interoperability are.
Annotation engine vs. ML engine¶
AnnoTrain is conceptually split into two systems, so a researcher can use it purely as an annotation/dataset platform without enabling deep learning.
Annotation engine: projects, media, contributors, tasks, classes, annotations, consensus, quality control, dataset creation.
ML engine: training jobs, inference jobs, evaluation, model versions, predictions, uncertainty, model publication. The dataset-export → preprocessing → training → evaluation pipeline is planned to be built with Snakemake for reproducibility, kept entirely on the ML-engine side - it does not replace or change the FastAPI web API used by the annotation engine.
An optional future project setting might be "automatically train a baseline model".
Open science and model sharing¶
A trained model is more than a .pt file. Each ModelVersion should eventually contain or reference: model architecture, weights, task type, input format/dimensions, preprocessing, output classes/format, training configuration, dataset version, evaluation metrics, date, software/code version, license, limitations, and a model card.
Possible model actions: Try model, Use for annotation, Download model, Fine-tune, Publish.
Potential future integrations (not required for the MVP): GitHub for source code, Hugging Face for model/dataset distribution, Zenodo for archival releases and citations.