Technical architecture

Note

This describes the target architecture. Only the parts marked as implemented in the Roadmap currently exist in the repository.

Overview

Browser
   │
   ▼
Frontend
   │
   ▼
FastAPI API
   ├────────► PostgreSQL
   │
   ├────────► Image / Object Storage
   │
   └────────► Job Queue
                    │
                    ▼
               ML Worker
                    │
                    ▼
                  GPU

The existing Vermeer Lab website (built with Quarto) remains the portal and documentation layer — it links out to the AnnoTrain application on a dedicated route or subdomain.

Backend

Initial choice: Python + FastAPI. The API is expected to eventually expose routes conceptually similar to:

GET  /projects
POST /projects

GET  /media/{id}

POST /annotations

POST /training-jobs
GET  /training-jobs/{id}

GET  /models/{id}

Database

Target database: PostgreSQL, storing metadata and annotations — not the original scientific images themselves.

Image storage

Scientific images live in file/object storage, initially potentially on the existing lab NAS. The backend stores references to these files rather than the files themselves. Long term, an S3-compatible abstraction would make the storage layer portable.

Compute

Training and inference run on separate workers with access to suitable CPU/GPU resources. The public API never directly exposes the GPU server.

Image handling

AnnoTrain does not send full-resolution scientific images to browsers unnecessarily. At project import:

Original image → Generate web representation → Store both

original/
web/

The web version is roughly 1600 pixels wide and compressed for interactive annotation; original images remain available for training, export, and high-resolution analysis.

Storage principles

Git must never be used for scientific datasets or trained model weights. The repository contains source code, documentation, configuration templates, tests, and lightweight examples only — large data stays in external storage.

Potential storage organization:

projects/
└── <project-id>/
    ├── original/
    ├── web/
    ├── annotations/
    ├── datasets/
    │   ├── v001/
    │   └── v002/
    └── models/
        ├── v001/
        └── v002/

Annotation engine vs. ML engine

AnnoTrain is conceptually split into two systems, so a researcher can use it purely as an annotation/dataset platform without enabling deep learning.

Annotation engine — projects, media, contributors, tasks, classes, annotations, consensus, quality control, dataset creation.

ML engine — training jobs, inference jobs, evaluation, model versions, predictions, uncertainty, model publication.

An optional future project setting might be "automatically train a baseline model".

Open science and model sharing

A trained model is more than a .pt file. Each ModelVersion should eventually contain or reference: model architecture, weights, task type, input format/dimensions, preprocessing, output classes/format, training configuration, dataset version, evaluation metrics, date, software/code version, license, limitations, and a model card.

Possible model actions: Try model, Use for annotation, Download model, Fine-tune, Publish.

Potential future integrations (not required for the MVP): GitHub for source code, Hugging Face for model/dataset distribution, Zenodo for archival releases and citations.