Defect and anomaly detection

Defect detection starts with naming the defects — scratch, crack, dent, corrosion, pit, void. A zero-shot detector takes those names as a prompt and finds the regions, without a labelled defect image to begin with.

The other half of the problem is the deviation that has no name yet. That case is handled by comparison: embed a reference set of acceptable parts, embed the part in front of the camera, and rank how close the two are.

The defect_detect_classify preset is the shortest path from an image to a defect report: text-prompted detection, a score filter, region crops, and a classification pass over each crop, with the results fused per defect.

• Detection prompts carry the class list — "scratch . crack . dent . corrosion" is a complete configuration.
• A filter at 0.3 by default, raised or lowered at the cost of missed defects or false alarms.
• Each detected region is cropped and passed through a classifier with the labels you supply, including a normal class for regions that turn out to be surface texture.
• Detections and classifications are combined into one result per defect, so a report line has a location and a type.

The class list is an argument, not a training set — a new defect type is a new word.

When acceptance depends on size rather than presence, the segmentation path returns a mask per defect. The grounding_dino_sam preset detects with text, filters at 0.25, prompts SAM with each box, and refines the mask with a morphology pass.

From a mask and its box, area can be computed in pixels and compared against a threshold the quality plan already uses. SAM variants let speed be traded for edge quality — base for a first version, large and huge when the boundary matters more than the clock.

MATA does not ship a dedicated anomaly classifier. The deviation case is composed from the embedding task and the recognition workflow: enrol a gallery of acceptable examples once, then match a new image by embedding similarity and treat the weakest matches as candidates for review.

CLIP embeddings run at 512 dimensions by default, and OSNet is available as a 256-dimension ONNX encoder for appearance work. What comes back is a ranked similarity, which a threshold turns into a flag — a triage aid rather than a verdict.

Numbers from the documented scenarios, for planning a first pilot rather than a specification to hold the product to:

• Defect detection and classification: about 200 ms per image on GPU with a 4 GB card, roughly five to ten times slower on CPU.
• Detection plus SAM segmentation and refinement: about 500 ms per image on GPU; the same measurement takes about 15 seconds per image on CPU.
• Thresholds are the main tuning dial: too low and ordinary surface texture appears as defects, too high and light scratches disappear.
• Thresholds can be overridden per call, so a stricter setting can gate a release and a looser one can feed a review queue.

A defect result is data before it is a picture. Labels, scores, boxes, masks and measured areas can be exported as JSON or CSV for the systems that already hold quality records, and rendered as annotated frames or crops for the people who need to see the part.

A review queue can therefore be split by confidence: borderline cases go to a person, clear cases are recorded, and the threshold that divides them is a setting rather than a rewrite.

• Structured exports for a quality system or a production record.
• Annotated frames and region crops for manual confirmation.
• Per-defect results that carry both a location and a type.

Begin with images that include both conforming and non-conforming parts, and write the defect names the way the detector wants to read them. Compare what comes back before choosing thresholds, and keep the images that disagree — they are the useful ones.

If the defect vocabulary is genuinely unknown, start from the reference-set route and treat the pipeline as triage. A labelled set gathered over a few weeks usually makes the prompt-based route sharper than it was on day one.