← all work deep-computer-vision

Anomaly detection

It produces a per-frame anomaly score, and the headline number says that score ranks anomalous

Result

featuresfrozen ImageNet ResNet-50 (`IMAGENET1K_V2`), 2048-d, nothing fine-tuned
supervision at trainvideo-level only — "this video contains an anomaly"
labels at testofficial per-frame temporal annotations, fetched at run time
annotation coverageevery official test video matched an annotation, and the folder label agreed with the annotation on every one of them (from the run log)
train / test videos1582 / 283, zero video-name overlap
test frames scored111134, of which 8520 anomalous (7.666%)
frame size64 × 64 — this dataset copy is decimated and small

The finding

AUC says it works, the operating points say it cannot be used

Anomalous frames are 7.666% of the test set. At the operating point that catches 80% of them, 43.754% of all frames are flagged — so the overwhelming majority of alarms are on normal footage, and raising recall to 0.90035 pushes false positives past half of everything.

A single AUC of 0.76329 reads as a working system. It is the same model at every threshold, and there is no threshold at which it is usable. That gap is the entire reason this write-up leads with the caveat rather than the metric: AUC is a ranking summary over a pooled frame set, and pooled ranking quality is not operational performance.

Elsewhere

Code and full results on GitHub

Every number is read from the file the run produced. github.com/hammasbuilds · kaggle.com/muhammadhammas13