Technical Diagnostics and Non-Destructive Testing, 2024, №03



2024 №03 (05)

DOI of Article 10.37434/tdnk2024.03.06

2024 №03 (07)

---

Technical Diagnostics and Non-Destructive Testing #3, 2024, pp. 39-44

Automated road surface diagnostic system with image complexing

D.V. Storozhyk, A.G. Protasov

NTUU «Igor Sikorsky Kyiv Polytechnic Institute». 37 Beresteysky Ave., 03056, Kyiv, Ukraine. E-mail: a.g.protasov@ gmail.com

The road surface diagnostic system is proposed. The basis of the system operation is processing images of defects that were obtained in the visible and infrared ranges of the spectrum. The system includes running laboratories on the car chassis that collect data from cameras, as well as image processing and decision support subsystems. The image processing subsystem provides conversion of the received images in the visible and infrared spectra to a format suitable for their complexation (fusion). The method of image fusion with adaptive determination of weights, which is implemented by a neural network, was chosen for the implementation of the complexing operation. When building this neural network, the principle of multimodal processing was applied, where each modality is represented using its own convolutional layers to highlight features that are evaluated by fully connected layers to determine weighting coefficients. After the completion of the complexing procedure, the obtained image is transferred to the decision support subsystem, which classifies the defects and establishes their geometric dimensions. To determine the dimensions, a convolutional neural model is used, which implements the image segmentation procedure. In the mechanism of logical conclusion, based on the model of presentation of knowledge obtained from the knowledge base, a conclusion is made regarding the level of defectiveness of the road section. The final element of the subsystem is software with a user interface that displays information obtained from past steps, a road passport, regulatory acts, information on past repair work, and data on budget support. 15 Ref., 1 Tabl., 7 Fig.
Keywords: automated diagnostic systems, neural networks, road surface

Received: 11.07.2024
Received in revised form: 15.08.2024
Accepted: 24.09.2024

References

1. Protasov, A.G., Storozhyk, D.V. (2023) Technologies for automation of road surface diagnostics. Vcheni Zapysky TNU, Vol. 34(73), 2, Pt. 1, 219-227. [in Ukrainian]. https://doi.org/10.32782/2663-5941/2023.2.1/35
2. Su, Y., Kang, S., Chang, J., Hsieh, S. (2010) Using dual lights for robotic pavement inspection. In: Proceedings of SICE Annual Conference, Vol. 1.
3. Bystrov, A., Hoart, E., Tran Thuy-Yung et al. (2018) Sensors for automotive remote road surface classification. IEEE International Conference on Vehicular Electronics and Safety. https://doi.org/10.1109/ICVES.2018.8519499
4. Elseicy, A., Alonso-Díaz, A., Solla, M. et al. (2022) Combined use of GPR and other NDTs for road pavement assessment: An Overview Remote Sens., 14, 4336. https://doi.org/10.3390/rs14174336
5. Fangyu, L., Liu, J., Wang, L. (2022) Asphalt pavement crack detection based on convolutional neural network and infrared thermography. IEEE Transactions on Intelligent Transportation Systems.
6. Sun, Z., Li, L., Liu, L. et al. (2022) On the importance of building high-quality training datasets for neural code search. Proceedings of the 44th International Conference on Software Engineering. https://doi.org/10.1145/3510003.3510160
7. Rad, R. (2024) Vision transformer for multispectral satellite imagery: Advancing landcover classification. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. https://doi.org/10.1109/WACV57701.2024.00799
8. Sham, J.F.C., Memon, S.A., Lo, T.Y. (2013) Application of continuous surface temperature monitoring technique for investigation of nocturnal sensible heat release characteristics by building fabrics in Hong Kong. Energy and Buildings, 58, 1-10. https://doi.org/10.1016/j.enbuild.2012.11.025
9. Solla, M., Lagüela, S., Gonzales-Jorge, H. et al. (2014) Approach to identify cracking in asphalt pavement using GPR and infrared thermographic methods: Preliminary findings. NDT&E Int., 62, 55-65. https://doi.org/10.1016/j.ndteint.2013.11.006
10. Storozhik, D.V., Protasov, A.G., Muraviov, O.V., et al. (2022) Automation of thermal non-destructive testing process by applying the method of complexing thermographs. Tekh. Diahnost. ta Neruiniv. Kontrol, 2, 20-23. [in Ukrainian]. https://doi.org/10.37434/tdnk2022.02.03
11. Zhang, Dengsheng (2019) Wavelet transform. Fundamentals of image data mining: Analysis, Features, Classification and Retrieval, 35-44. https://doi.org/10.1007/978-3-030-17989-2_3
12. Storozhyk, D.V., Protasov, A.G. (2022) Image processing technologies based on complexing data (Review). Tekh. Diahnost. ta Neruiniv. Kontrol, 4, 17-26. [in Ukrainian]. https://doi.org/10.37434/tdnk2022.04.03
13. Eisler, K., Homma, C., Goldammer, M. et al. (2013) Fusion of visual and infrared thermography images for advanced assessment in non-destructive testing. Review Science Instruments, 84, 064902. https://doi.org/10.1063/1.4808280
14. Changqi Sun, Cong Zhang, Naixue Xiong (2020) Infrared and Visible Image Fusion Techniques Based on Deep Learning: A Review. Electronics, 9, 21-62. https://doi.org/10.3390/electronics9122162
15. Linnyk, M.O., Nakonechnyi, A.Y. (2023) Methods and algorithms for complexing images and thermal signals. Computer systems and networks, 5(1), 60-71. [in Ukrainian]. https://doi.org/10.23939/csn2023.01.060

---

Advertising in this issue:

---