An end-to-end computer vision methodology for quantitative metallography

Matan Rusanovsky, Ofer Beeri, Gal Oren

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


Metallography is crucial for a proper assessment of material properties. It mainly involves investigating the spatial distribution of grains and the occurrence and characteristics of inclusions or precipitates. This work presents a holistic few-shot artificial intelligence model for Quantitative Metallography, including Anomaly Detection, that automatically quantifies the degree of the anomaly of impurities in alloys. We suggest the following examination process: (1) deep semantic segmentation is performed on the inclusions (based on a suitable metallographic dataset of alloys and corresponding tags of inclusions), producing inclusions masks that are saved into a separated dataset. (2) Deep image inpainting is performed to fill the removed inclusions parts, resulting in ‘clean’ metallographic images, which contain the background of grains. (3) Grains’ boundaries are marked using deep semantic segmentation (based on another metallographic dataset of alloys), producing boundaries that are ready for further inspection on the distribution of grains’ size. (4) Deep anomaly detection and pattern recognition is performed on the inclusions masks to determine spatial, shape, and area anomaly detection of the inclusions. Finally, the end-to-end model recommends an expert on areas of interest for further examination. The physical result can re-tune the model according to the specific material at hand. Although the techniques presented here were developed for metallography analysis, most of them can be generalized to a broader set of microscopy problems that require automation. All source-codes as well as the datasets that were created for this work, are publicly available at

Original languageEnglish
Article number4776
JournalScientific Reports
Issue number1
StatePublished - 1 Dec 2022
Externally publishedYes

ASJC Scopus subject areas

  • General


Dive into the research topics of 'An end-to-end computer vision methodology for quantitative metallography'. Together they form a unique fingerprint.

Cite this