MAEDAY: MAE for few- and zero-shot AnomalY-Detection

Eli Schwartz; Assaf Arbelle; Leonid Karlinsky; Sivan Harary; Florian Scheidegger; Sivan Doveh; Raja Giryes

doi:10.1016/j.cviu.2024.103958

MAEDAY: MAE for few- and zero-shot AnomalY-Detection

Eli Schwartz, Assaf Arbelle, Leonid Karlinsky, Sivan Harary, Florian Scheidegger, Sivan Doveh, Raja Giryes

Research output: Contribution to journal › Article › peer-review

8 Scopus citations

Abstract

We propose using Masked Auto-Encoder (MAE), a transformer model self-supervisedly trained on image inpainting, for anomaly detection (AD). Assuming anomalous regions are harder to reconstruct compared with normal regions. MAEDAY is the first image-reconstruction-based anomaly detection method that utilizes a pre-trained model, enabling its use for Few-Shot Anomaly Detection (FSAD). We also show the same method works surprisingly well for the novel tasks of Zero-Shot AD (ZSAD) and Zero-Shot Foreign Object Detection (ZSFOD), where no normal samples are available.

Original language	English
Article number	103958
Journal	Computer Vision and Image Understanding
Volume	241
DOIs	https://doi.org/10.1016/j.cviu.2024.103958
State	Published - 1 Apr 2024
Externally published	Yes

Keywords

Anomaly-detection
Foreign object detection
Masked autoencoder

ASJC Scopus subject areas

Software
Signal Processing
Computer Vision and Pattern Recognition

Access to Document

10.1016/j.cviu.2024.103958

Cite this

@article{9c730a65794748a7b3465abb001021ee,

title = "MAEDAY: MAE for few- and zero-shot AnomalY-Detection",

abstract = "We propose using Masked Auto-Encoder (MAE), a transformer model self-supervisedly trained on image inpainting, for anomaly detection (AD). Assuming anomalous regions are harder to reconstruct compared with normal regions. MAEDAY is the first image-reconstruction-based anomaly detection method that utilizes a pre-trained model, enabling its use for Few-Shot Anomaly Detection (FSAD). We also show the same method works surprisingly well for the novel tasks of Zero-Shot AD (ZSAD) and Zero-Shot Foreign Object Detection (ZSFOD), where no normal samples are available.",

keywords = "Anomaly-detection, Foreign object detection, Masked autoencoder",

author = "Eli Schwartz and Assaf Arbelle and Leonid Karlinsky and Sivan Harary and Florian Scheidegger and Sivan Doveh and Raja Giryes",

note = "Publisher Copyright: {\textcopyright} 2024 Elsevier Inc.",

year = "2024",

month = apr,

day = "1",

doi = "10.1016/j.cviu.2024.103958",

language = "English",

volume = "241",

journal = "Computer Vision and Image Understanding",

issn = "1077-3142",

publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - MAEDAY

T2 - MAE for few- and zero-shot AnomalY-Detection

AU - Schwartz, Eli

AU - Arbelle, Assaf

AU - Karlinsky, Leonid

AU - Harary, Sivan

AU - Scheidegger, Florian

AU - Doveh, Sivan

AU - Giryes, Raja

PY - 2024/4/1

Y1 - 2024/4/1

N2 - We propose using Masked Auto-Encoder (MAE), a transformer model self-supervisedly trained on image inpainting, for anomaly detection (AD). Assuming anomalous regions are harder to reconstruct compared with normal regions. MAEDAY is the first image-reconstruction-based anomaly detection method that utilizes a pre-trained model, enabling its use for Few-Shot Anomaly Detection (FSAD). We also show the same method works surprisingly well for the novel tasks of Zero-Shot AD (ZSAD) and Zero-Shot Foreign Object Detection (ZSFOD), where no normal samples are available.

AB - We propose using Masked Auto-Encoder (MAE), a transformer model self-supervisedly trained on image inpainting, for anomaly detection (AD). Assuming anomalous regions are harder to reconstruct compared with normal regions. MAEDAY is the first image-reconstruction-based anomaly detection method that utilizes a pre-trained model, enabling its use for Few-Shot Anomaly Detection (FSAD). We also show the same method works surprisingly well for the novel tasks of Zero-Shot AD (ZSAD) and Zero-Shot Foreign Object Detection (ZSFOD), where no normal samples are available.

KW - Anomaly-detection

KW - Foreign object detection

KW - Masked autoencoder

UR - http://www.scopus.com/inward/record.url?scp=85185393889&partnerID=8YFLogxK

U2 - 10.1016/j.cviu.2024.103958

DO - 10.1016/j.cviu.2024.103958

M3 - Article

AN - SCOPUS:85185393889

SN - 1077-3142

VL - 241

JO - Computer Vision and Image Understanding

JF - Computer Vision and Image Understanding

M1 - 103958

ER -

MAEDAY: MAE for few- and zero-shot AnomalY-Detection

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this