Anomaly detection for the visual inspection of industrial and biological data

Abstract

Automated visual inspection embodies a markedly significant discipline across diverse technical fields, featuring a potential to retrieve a high amount of information from a single image and to process multiple images efficiently. Such methods usually involve the traditional and deep learning computer vision techniques to perform the classification and object detection tasks. With the recent progress in artificial neural networks and computing, many applications are solved using deep learning. Although effective, deep learning usually requires a high amount of annotated images to train the models; such images typically represent all possible situations, including defective observations. This task, however, can be challenging due to a lack of defective cases or expensive collection of such data. These issues are eliminable via anomaly detection, where the methods are trained with zero or only a small number of defective samples. In the training, the relevant detection techniques rely on datasets with zero or only a minor quantity of defective (anomalous) images, and they extract a representation of the normal data. Such approach brings many benefits as an easier collection of the training datasets, robustness against unknown anomalies and in some cases, leaving out the annotation process.

The research presented herein comprises two methods for solving the visual anomaly detection task, four novel datasets, and two data collection platforms. One of the methods exploits deep learning object detectors and the common object detector models YOLO, SSD, and Faster R-CNN to reveal the anomalies by using various annotation approaches. This approach proved to be suitable for datasets comprising a sufficient amount of anomalies with a similar appearance; this capability is indispensable for any successful training of such detectors. The other technique then involves reconstructing the input images to emphasise anomalies followed by several feature extraction techniques and one-class classifiers. This option was demonstrated as more convenient for datasets that exhibit a limited amount of anomalous images available for the training or a high variance of the anomalous classes.

Two of the presented datasets cover the industrial domain, and these are followed by two biological ones, which relate to phytoplankton and bee parasites. The proposed approaches were compared on the phytoplankton anomaly dataset. The object detector-based option achieved better results over all the phytoplankton species, reaching the F1- score of 0.86 using the normal samples, and the anomalies annotation procedure reached an above-the-average F1-score of 0.75 using the reconstruction based approach. The reconstruction-based approach performed better in the case of species-specific experiment while reaching an average F1-score of 0.85 over the average F1-score of 0.62 in the case of the object detector-based approach.

The data collection platforms include a bee health inspection device and an anomaly detection demonstration platform. The device was designed as an open project to facilitate long-term monitoring, involving 3D printed parts and commonly available electronic modules. Using this device, two other datasets planned to be employed in the future research were collected.