Technical Diagnostics and Non-Destructive Testing, 2023, №02



2023 №02 (04)

DOI of Article 10.37434/tdnk2023.02.05

2023 №02 (06)

---

Technical Diagnostics and Non-Destructive Testing #2, 2023, pp. 34-40

Automation of the process of segmentation of images of metal surface defects using the neural network U-Net

Y.V. Steshenko, A.S. Momot, A.G. Protasov O.V. Muraviov

National Technical University of Ukraine «Igor Sikorsky Kyiv Polytechnic Institute». 37 Beresteysky Avenue, 03056, Kyiv, Ukraine. E-mail: yaroslav.steshenko@ukr.net; drewmomot@gmail.com

The paper deals with the task of automated segmentation of images of defects metal surfaces. The aim of the study is to improve segmentation algorithms using deep learning methods. The expediency of using the U-Net neural network, which is effective in the tasks of semantic image segmentation, is substantiated. With the help of a special architecture, the network can create segmentation masks with high efficiency. The training dataset for the neural network contained images of four classes of defects, including chips, cracks, and stains. As a result of analyzing the distribution of defect classes in the training dataset, it was concluded that the classes were unbalanced, which negatively affects the training results. To evaluate the quality of network training, a set of metrics such as Accuracy, F1 Score, and IOUScore is considered. The feasibility of using these metrics is analyzed, taking into account the features of the training data set. It is proved that under conditions of significant imbalance of classes, the Accuracy metric does not reflect the real quality of the model. The influence of different variants of the ResNet architecture backbone on the training results is analyzed. It is determined that the best results are shown by the ResNet18 model, which managed to obtain a Dice coefficient of 69 % and an IOUScore of 53 % on the test data set. It is proved that an increase in the number of model parameters does not always lead to an improvement in the reliability of the results. The article provides examples of test images and defect masks and countures predicted by the neural network. 13 Ref., 1 Tabl., 7 Fig.
Keywords: metal surfaces, image segmentation, neural networks

Received: 12.04.2023

References

1. Storozhyk, D.V., Protasov, A.G. (2022) Image processing technologies based on complexing data. Tekh. Diagnost. ta Neruiniv. Kontrol, 4, 17-26 [in Ukrainian]. https://doi.org/10.37434/tdnk2022.04.03
2. Skladchykov, I.O., Momot, A.S., Galagan, R.M., Bohdan, G.A., Trotsiuk, K.M. (2022) Application of YOLOX deep learning model for automated object detection on thermograms. Information Extraction and Process, 50, 69-77. https://doi.org/10.15407/vidbir2022.50.069
3. Piironen, T., Silven, O., Pietikäinen, M., Laitinen, T., Strömmer, E. (1990) Automated Visual Inspection of Rolled Metal Surfaces. Machine Vision and Applications, 3, 247-254. https://doi.org/10.1007/BF01211850
4. Vorobel, R., Ivasenko, I., Berehulyak, O., Mandzii, T. (2021) Segmentation of rust defects on painted steel surfaces by intelligent image analysis. Automation in Construction, 123, 103515. https://doi.org/10.1016/j.autcon.2020.103515
5. Aslam, Y., Santhi, N., Ramasamy, N., Ramar, K. (2020) Localization and segmentation of metal cracks using deep learning. Journal of Ambient Intelligence and Humanized Computing, 12, 4205-4213. https://doi.org/10.1007/s12652-020-01803-8
6. Sharma, M., Lim, J., Lee, H. (2022) The Amalgamation of the Object Detection and Semantic Segmentation for Steel Surface Defect Detection. Applied Sciences, Vol. 12, Issue 12, 6004. https://doi.org/10.3390/app12126004
7. Ronneberger, O., Fischer P., Brox, T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. Lecture Notes in Computer Science, Springer, Cham. https://doi.org/10.1007/978-3-319-24574-4_28
8. Steshenko, Y.V., Momot, A.S. (2022) Review of the effectiveness of neural network architectures for automated image segmentation. In: Proc. of XVIII All-Ukrainian scientific and practical conference of students, graduate students and young scientists «Efficiency and automation of engineering solutions in instrument construction» (Kyiv, Ukraine, 06-07 December 2022), KPI, 226-229 [in Ukrainian].
9. Xiao, L., Zhu, Y., Punithakumar, K., Le, L.H., Li, B. (2020). Esophagus Segmentation in Computed Tomography Images Using a U-Net Neural Network with a Semiautomatic Labeling Method. IEEE Access, 8, 202459-202468. https://doi.org/10.1109/ACCESS.2020.3035772
10. (2020) Severstal: Steel Defect Detection. Kaggle. Available at: https://www.kaggle.com/competitions/severstal-steel-defect-detection/data (Accessed: April 3, 2023).
11. He, H., Garcia, E.A. (2009) Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering, Vol. 21(9), 1263-1284. https://doi.org/10.1109/TKDE.2008.239
12. Maxwell, A.E., Bester, M.S., Guillen, L.A., Ramezan, C.A., Carpinello, D.J., Fan, Y., Hartley, F.M., Maynard, S.M., Pyron, J.L. (2020) Semantic Segmentation Deep Learning for Extracting Surface Mine Extents from Historic Topographic Maps. Remote Sensing, Vol. 12, Issue 24, 4145. https://doi.org/10.3390/rs12244145
13. Benjdira, B., Ammar, A., Koubaa, A., Ouni, K. (2020) Data-Efficient Domain Adaptation for Semantic Segmentation of Aerial Imagery Using Generative Adversarial Networks. Applied Sciences, Vol. 10, Issue 3, 1092. https://doi.org/10.3390/app10031092

---

Advertising in this issue:

---