Pavement Distress Detection From Orthophotos With Two-Stream Convolutional Neural Networks

Abstract

Automated pavement distress detection is an important but challenging task towards the goal of timely road maintenance. Given the vastness of road networks across the world, there is a lot of labor involved in manual defect detection for roads. In the recent years, however, convolutional neural networks have been shown to achieve groundbreaking results in the field of image classification. This thesis seeks to research and develop methods for applying convolutional neural networks to pavement distress detection for sections of orthophotos (orthoframes) with a large resolution. To address GPU memory limitations and increase detection localization, a sliding-window approach is used to partition the orthoframe into 224x224-pixel segments, which are subject to binary classification. However, the sliding-window approach does not allow for the model to account for the context surrounding the segment and results may suffer due to the small window size. This thesis proposes a ResNet architecture based convolutional neural network which accounts for two inputs streams, one of which is the 224x224-pixel content segment, which is subject to classification, and the other is the downscaled context view around the content segment. Experiments on two different datasets show an increased classification accuracy for the two-stream approach compared to the single stream approach.

Related publications

• R. Lõuk, A. Tepljakov, and A. Riid, “A Two-Stream Context-Aware ConvNet for Pavement Distress Detection,” in 2020 43rd International Conference on Telecommunications and Signal Processing (TSP). IEEE, 2020, pp. 270–273. [DOI] [Bib]
• R. Lõuk, A. Riid, R. Pihlak, and A. Tepljakov, “Pavement Defect Segmentation in Orthoframes with a Pipeline of Three Convolutional Neural Networks,” Algorithms, vol. 13, no. 8, pp. 198, 2020. [DOI] [Bib]
• A. Riid, R. Lõuk, R. Pihlak, A. Tepljakov, and K. Vassiljeva, “Pavement Distress Detection with Deep Learning Using the Orthoframes Acquired by a Mobile Mapping System,” Applied Sciences, vol. 9, no. 22, pp. 4829, 2019. [DOI] [Bib]
• A. Tepljakov, A. Riid, R. Pihlak, K. Vassiljeva, and E. Petlenkov, “Deep Learning for Detection of Pavement Distress using Nonideal Photographic Images,” IEEE, 2019. [DOI] [Bib]