Segmentation and classification of dot and non-dot-like fluorescence in situ hybridization signals for automated detection of cytogenetic abnormalities

Signal segmentation and classification of fluorescence in situ hybridization (FISH) images are essential for the detection of cytogenetic abnormalities. Since current methods are limited to dot-like signal analysis, we propose a methodology for segmentation and classification of dot and non-dot-like signals. First, nuclei are segmented from their background and from each other in order to associate signals with specific isolated nuclei. Second, subsignals composing non-dot-like signals are detected and clustered to signals. Features are measured to the signals and a subset of these features is selected representing the signals to a multiclass classifier. Classification using a naïve Bayesian classifier (NBC) or a multilayer perceptron is accomplished. When applied to a FISH image database, dot and non-dot-like signals were segmented almost perfectly and then classified with accuracy of ∼ 80% by either of the classifiers.