Safety monitoring

Safety monitoring applies the same pipeline machinery as inspection to a different question: not whether the product is correct, but whether the place is behaving the way it should.

It runs on site hardware, processes streams where they are produced, and produces events that route to the people who can act on them.

• Checks run continuously rather than in a periodic manual sweep.
• The same frame can answer several questions at once, because each check is a node in one graph.
• Every event carries its evidence, so a decision can be explained afterwards.

Person detection plus multi-object tracking turns a frame-by-frame stream into movements with persistent identities. The crowd_monitoring preset is the documented composition, and it produces counts rather than images.

• Detection with a standard detector such as facebook/detr-resnet-50, filtered to person-class detections.
• ByteTrack for fixed camera positions; BotSort when the camera pans or the platform moves, since it compensates for camera motion.
• Unique track IDs give a headcount, which a rule compares against a density threshold to raise an alert.
• Appearance re-identification can recover an identity after an occlusion, using a CLIP or OSNet encoder.

The zero-shot path covers checks that are awkward to train: name what you are looking for — a helmet, a high-visibility vest, a barrier, a person inside a marked area — and the detector looks for it in the frame without any example images.

For objects rather than people, the suspicious-object scenario chains three stages: text-prompted detection finds candidates such as a bag or a package, SAM gives them clean boundaries, and a vision-language model describes the state and the surroundings so a person can judge the context.

A new check is a new prompt, which means a safety review can change its mind faster than a training cycle.

The vlm_scene_understanding preset asks a vision-language model to describe what a camera sees while other tasks gather evidence in parallel.

• The description prompt can be aimed at the question that matters — unusual behaviour, a blocked route, a condition worth reporting.
• Entity detection runs alongside it, so the description is accompanied by located objects rather than standing alone.
• Depth estimation adds scene geometry where the layout of the space matters.
• All three are fused into one result per frame, so an event carries a narrative and the detections behind it.

Live sources are handled as streams. RTSP feeds and long videos run with constant memory, and frame policies decide what gets processed: every Nth frame, always the latest frame for real-time work, or a bounded queue that drops frames under load rather than falling behind.

Events route onward in the formats a site already uses — JSON or CSV to a system, annotated frames and crops to a person — and each one is recorded, so a decision can be reviewed after the fact.

• Frame policies keep the compute budget fixed when camera count grows.
• Constant-memory streaming means a long shift does not exhaust the machine.
• Records per event support review rather than real-time reaction alone.

A monitoring pipeline produces candidates; a person decides. How useful it is depends on where the thresholds sit, and safety decisions should not be handed to a threshold without review.

The heavier checks need a GPU to run at useful rates — anything involving a vision-language model in particular. Detection and tracking alone are documented as workable on CPU at reduced frame rates.