Edge deployment

Edge deployment here is a property of how a pipeline is loaded and run, not a separate product. A site machine loads models from local files, executes on whatever compute it has, and needs no route to the public internet to keep working.

That matters for the same reason the processing is on site in the first place: the frames, the crops and the events stay inside the network, and the system keeps running when the connection does not.

The universal loader is the mechanism: point it at a file instead of a hub ID and the matching runtime adapter takes over, with the pipeline above it unchanged.

• ONNX Runtime covers detection, classification and embedding, so an ONNX detector can be loaded by path and served by that runtime.
• TorchScript archives cover detection and classification for teams already exporting that way.
• Torchvision's detection family — RetinaNet, Faster R-CNN, FCOS, SSD — runs locally when a lighter or more familiar backbone is the priority.
• Paths, thresholds and per-task defaults live in .mata/models.yaml, so a site's model set is a configuration file rather than code.

Documented throughput on one RTX 3080-class card: SSD at roughly 60 FPS, RT-DETR at roughly 50 FPS, RetinaNet at roughly 40 FPS, FCOS at roughly 30 FPS and DETR at roughly 20 FPS. On CPU the same work runs five to ten times slower, which suits lower frame rates and event-style processing rather than continuous video.

Model size is the other lever: the GroundingDINO tiny checkpoint is 48M parameters, SmolVLM is built for edge and mobile use, and OSNet is available as a 256-dimension ONNX export for appearance work. Pick the smallest model that answers the question, then check the rate.

Decide the frames per second per camera first — the hardware follows from that number, and so does the model list.

Weights are fetched once and cached locally; after that the machine runs without reaching the public internet. MATA does not distribute model weights — they come from their own sources under their own licences — which is worth knowing when a site is air-gapped or a licence needs review.

A CPU-only installation is a first-class path: install the CPU build of PyTorch, then datamata, and the same pipeline runs without a GPU. The pipeline code does not change between the two.

Cameras on site produce streams that a small machine still has to keep up with, and there are explicit answers for that.

• Constant-memory streaming handles RTSP sources and long videos, so a full shift does not exhaust the machine.
• Frame policies — every Nth frame, always the latest frame, or a bounded queue — decide how much compute each camera is allowed.
• Tracking is fully inside the framework: ByteTrack and BotSort are vendored, so there is no external tracking service to install or reach.
• Detections, masks and events can be written to local formats or a local store, keeping the whole loop on site.

One machine rarely has to do everything. A usual split puts detection, classification and tracking on the box beside the cameras, where the frames already are, and sends only the results — or the crops that need a second opinion — to a machine with a GPU for the heavier assessment.

Because loading is uniform, that split is a decision about strings rather than about code: the same pipeline definition runs on both machines, with different model sources in each configuration file.

• Site box: detection, tracking and rule evaluation at the frame rate it can hold.
• Central machine: the vision-language checks where a GPU is available.
• One configuration per machine, one pipeline definition between them.

Today the portable path is the artefacts you bring: ONNX and TorchScript files, loaded through the same adapters as everything else. The mata export command is a stub at present, and quantized ONNX export, TensorRT and TFLite export are on the v2.0 roadmap rather than in the shipped release.

For planning that means one honest caveat: a site that needs a quantized model or a TensorRT engine today has to produce that artefact outside the framework and load the result back in.