System for detecting crack growth of asphalt pavement based on binocular image analysis

Abstract

A binocular image analysis-based asphalt road surface damage detection system, comprising five sub-systems for crack development degree detection model establishment, information collection, information analysis, information transmission and information distribution. Specifically, road surface damage information detection is performed by means of a mobile platform, a processing end, a temperature measurement-type infrared thermal imaging instrument, an ordinary image collector, a vibration sensor, image processing technology, edge cloud computing technology, an embedded system, system integration technology and real-time positioning technology.

Claims

1. A system for detecting crack growth degree of an asphalt pavement based on binocular image analysis, the system comprising: a thermometric infrared camera for capturing infrared images of the asphalt pavement that comprise information of an atmospheric temperature; an active infrared image sensor for capturing ordinary images of the asphalt pavement; a vehicle positioning device for determining spatial and temporal coordinate TS.sub.1, TS.sub.2, TS.sub.3; a three-axis acceleration sensor for collecting vibration information of a rear axle of a vehicle; a mobile platform onto which the thermometric infrared camera, the active infrared image sensor, the vehicle positioning device and the three-axis acceleration sensor are fixed, the mobile platform being provided on the vehicle to continuously move with the vehicle; and one or more processors configured to perform: a) collecting at least ten sample infrared images of cracks on the asphalt pavement, recording a sample atmospheric temperature and a crack growth degree; ranking the crack growth degree into three levels of 1, 2 and 3 according to a traditional crack severity classification method; b) processing the sample infrared images in step a), cutting off transition zone between said crack samples and said asphalt pavement and obtaining measured temperature difference data between said crack samples and said asphalt pavement in said sample infrared images; c) based on the measured temperature difference data in step b), the atmospheric temperature and said crack growth degree in step a), obtaining two classification functions by Support Vector Machine: ΔT.sub.12=a.sub.12T+b.sub.12 and ΔT.sub.23=a.sub.23T+b.sub.23, where T (° C.) is an independent variable, representing the atmospheric temperature, ΔT.sub.12 and ΔT.sub.23 are dependent variables, representing the reference temperature difference data; h) matching the infrared images, the atmospheric temperature and the ordinary images with spatial and temporal coordinate TS.sub.1 and TS.sub.2 to obtain disease information; i) processing the vibration information of the rear axle of the vehicle and matching said vibration information of the rear axle with the spatial and temporal coordinate TS.sub.3 to obtain roughness information; j) uploading the disease information and the roughness information to a server database; and k) displaying the disease information and the roughness information of the server database, and identifying and locating crack areas on the asphalt pavement based on the disease information and the roughness information.

2. The system according to claim 1, wherein step h) includes the following sub-steps: h.sub.1) processing ordinary images by algorithm for pavement disease identification model to obtain the presence and type of the disease; h.sub.2) processing infrared images by algorithm for pavement disease identification model to obtain crack area; h.sub.3) processing infrared images by algorithm for pavement disease identification model to obtain crack growth degree; h.sub.4) labeling spatial and temporal coordinate TS.sub.1 and TS.sub.2 with the presence and type of the disease, crack area and crack growth degree.

3. The system according to claim 2, wherein step i) includes the following sub-steps: i.sub.1) processing vibration information by the roughness detection algorithm based on power spectral density to obtain the roughness information of the whole road section; i.sub.2) labeling spatial and temporal coordinate TS.sub.3 with the roughness information said in i.sub.1, where time coordinates are up to minutes and space coordinates to meters.

4. The system according to claim 1, wherein sub-step h.sub.3) includes: h.sub.31) matching the infrared images of the pavement with the legend to obtain the measured temperature values of the pavement T.sub.0; h.sub.32) dividing the crack area into p (p≥2) segments of the length l.sub.1 . . . l.sub.p and matching each segment with legend to obtain the measured temperature T.sub.1 . . . T.sub.p; h.sub.33) subtracting T.sub.0 from the measured temperature T.sub.1 . . . T.sub.p to obtain the measured temperature difference of each segment ΔT.sub.1 . . . ΔT.sub.p; h.sub.34) calculating the weighted mean according to the following formula to obtain a final measured temperature difference ΔT; $\Delta T=\frac{T_1{l}_1+\mathrm{.Math.}+T_p{l}_p}{l_1+\mathrm{.Math.}+l_p}$ h.sub.35) introducing atmospheric temperature into the classification function ΔT.sub.12=a.sub.12T+b.sub.12 to obtain the reference temperature difference ΔT.sub.12; h.sub.36) introducing atmospheric temperature into the classification function ΔT.sub.23=a.sub.23T+b.sub.23 to obtain the reference temperature difference ΔT.sub.23; h.sub.37) comparing ΔT with ΔT.sub.12 and ΔT.sub.23; if ΔT≤ΔT.sub.12, crack growth degree is 1; if ΔT.sub.12≤ΔT≤ΔT.sub.23, crack growth degree is 2; and if ΔT≥ΔT.sub.23, crack growth degree is 3.

5. The system according to claim 4, wherein step i) includes the following sub-steps: i.sub.1) processing vibration information by the roughness detection algorithm based on power spectral density to obtain the roughness information of the whole road section; i.sub.2) labeling spatial and temporal coordinate TS.sub.3 with the roughness information said in i.sub.1, where time coordinates are up to minutes and space coordinates to meters.

6. The system according to claim 1, wherein step i) includes the following sub-steps: i.sub.1) processing vibration information by the roughness detection algorithm based on power spectral density to obtain the roughness information of the whole road section; i.sub.2) labeling spatial and temporal coordinate TS.sub.3 with the roughness information said in ii, where time coordinates are up to minutes and space coordinates to meters.

7. The system according to claim 1, wherein step i) includes the following sub-steps: i.sub.1) processing vibration information by the roughness detection algorithm based on power spectral density to obtain the roughness information of the whole road section; i.sub.2) labeling spatial and temporal coordinate TS.sub.3 with the roughness information said in ii, where time coordinates are up to minutes and space coordinates to meters.

8. The system according to claim 1, wherein step i) includes the following sub-steps: i.sub.1) processing vibration information by the roughness detection algorithm based on power spectral density to obtain the roughness information of the whole road section; i.sub.2) labeling spatial and temporal coordinate TS.sub.3 with the roughness information said in i.sub.1, where time coordinates are up to minutes and space coordinates to meters.

9. The system according to claim 1, wherein step i) includes the following sub-steps: i.sub.1) processing vibration information by the roughness detection algorithm based on power spectral density to obtain the roughness information of the whole road section; i.sub.2) labeling spatial and temporal coordinate TS.sub.3 with the roughness information said in i.sub.1, where time coordinates are up to minutes and space coordinates to meters.

10. The system according to claim 1, wherein step i) includes the following sub-steps: i.sub.1) processing vibration information by the roughness detection algorithm based on power spectral density to obtain the roughness information of the whole road section; i.sub.2) labeling spatial and temporal coordinate TS.sub.3 with the roughness information said in i.sub.1, where time coordinates are up to minutes and space coordinates to meters.

11. The system according to claim 1, wherein step k) includes the following sub-steps: k.sub.1) displaying the disease information and roughness information stored in the server database a in the visual interface through the form of category and time series log; k.sub.2) map visualizing the disease information and roughness information stored in the server database by category and spatial sequence.

12. The system according to claim 11, wherein: in sub-step k.sub.1): the said log form can realize classified querying and sorting by time and space; the said log form can import external disease information and realize saving and calling; the said log supports report generation, statistical chart visualization and analysis model access; in sub-step k.sub.2): the said map visualization can be classified, viewed and sorted by space; the said map visualization supports single point and single disease query and generates report forms.

13. The system according to claim 1, wherein step k) includes the following sub-steps: k.sub.1) displaying the disease information and roughness information stored in the server database a in the visual interface through the form of category and time series log; k.sub.2) map visualizing the disease information and roughness information stored in the server database by category and spatial sequence.

14. The system according to claim 1, wherein step k) includes the following sub-steps: k.sub.1) displaying the disease information and roughness information stored in the server database a in the visual interface through the form of category and time series log; k.sub.2) map visualizing the disease information and roughness information stored in the server database by category and spatial sequence.

15. The system according to claim 1, wherein step k) includes the following sub-steps: k.sub.1) displaying the disease information and roughness information stored in the server database a in the visual interface through the form of category and time series log; k.sub.2) map visualizing the disease information and roughness information stored in the server database by category and spatial sequence.

16. The system according to claim 1, wherein step k) includes the following sub-steps: k.sub.1) displaying the disease information and roughness information stored in the server database a in the visual interface through the form of category and time series log; k.sub.2) map visualizing the disease information and roughness information stored in the server database by category and spatial sequence.

Description

FIGURE INSTRUCTION

(1) FIG. 1 is the schematic diagram of model formulation of system sampling data, including infrared image (101), atmospheric temperature (102), crack growth degree (103), measured temperature difference data (104);

(2) FIG. 2 is the schematic diagram of the fixing collection device;

(3) FIG. 3 is the schematic diagram of the acquisition terminal and the processing layer;

(4) FIG. 4 is schematic diagram of temperature difference in infrared thermal image;

(5) FIG. 5 is a roadmap of image processing technology;

(6) FIG. 6 is the image processing diagram;

(7) FIG. 7 shows the relationship between illumination and temperature difference;

(8) FIG. 8 shows the relationship between atmospheric temperature and temperature difference;

(9) FIG. 9 is a support vector machine linear classification diagram;

(10) FIG. 10 depicts the asphalt pavement crack area, pavement area and transition interference area;

(11) FIG. 11 is a schematic diagram of temperature differences obtained from different crack sections;

(12) FIG. 12 depicts the application phase samples data, including infrared image (201), atmospheric temperature (202), ordinary image (203), vibration information (204), spatial and temporal coordinates TS.sub.1, TS.sub.2, and TS.sub.3.

IMPLEMENTATION

(13) (1) Model Formulation

(14) Considering the differences in climatic conditions in different regions, the relationship between the temperature difference of crack and pavement and the growth degree works in certain region. First, a certain number of infrared image acquisition of cracks in a suitable collection environment is performed. Then, SVM is carried out according to the atmospheric temperature, the temperature difference between the pavement and the crack and the crack growth degree to obtain the classification function. Then the detection model in a certain season in this region is formulated.

(15) (2) Environment Determination

(16) The predictive relationship model works under certain environmental conditions to ensure accuracy. The test environment can be summarized as follows: a) Sunny day b) Effective time for data collection is from 8 am to 4 pm. c) The pavement is completely dry. d) The pavement is clean.
(3) Data Acquisition

(17) The ordinary camera (with night vision function) and the infrared camera (thermal measurement) are fixed on the mobile platform. The pavement inspection is carried out in the determined environment. First, a specific lane is selected to record the initial lane information, and the inspection speed can reach up to 80 km/h, two cameras are used for real-time image acquisition during the inspection. The frequency is greater than or equal to 60 Hz. The vibration sensor acquisition frequency is at least 100 Hz. The real-time high-frequency sampling is performed. At the same time, the positioning device and system time are used for real-time spatial and temporal coordinate acquisition. The application scenario is asphalt pavement for various grades of road. Here is the application phase, and the specific sampling data is shown in FIG. 12.

(18) (4) Data Processing

(19) Firstly, real-time processing of two images is carried out. Each image is treated with a road disease identification algorithm, and the images are classified and stored in diseased manner and disease-free manner. Image in diseased manner is further identified for disease type. Image with crack is further identified for the growth degree based on the infrared image. Images with crack are collected to form the log information, that is matched with the position information. The same crack may exist in multiple images. After processing, each image will receive disease information. After matching with the position information, the disease can be determined. The uniqueness is that there is only one disease type at the same position. Finally, the disease information is uploaded to the central server for display and application.

(20) At the same time, the vibration signal is processed, and the roughness information of the road segment is calculated in real time through the high-frequency vibration signal, and matched with the spatial and temporal coordinates to form a complete roughness detection information, which is then uploaded to the server database for display and application.

(21) (5) Data Release

(22) Data release is divided into two levels. The first layer is the display of the working layer. The data release layer is configured on the vehicle side. The display terminal is directly connected to the processing terminal. The data is derived from the real-time processing terminal, mainly to visualize the data stream and the working status without any interactive interface. The working conditions are divided into normal, abnormal and error. The display of the data flow assists the relevant personnel to eliminate the abnormal working conditions. The wrong working condition is a device problem and needs to be checked for downtime. The abnormal working conditions are mainly the abnormal collection conditions such as unsatisfied climatic conditions or too fast moving of platform.

(23) The second layer is the release of the application layer, mainly to visualize Web form, which is divided into log interface and map visualization interface. The log page can realize the operation of querying, exporting and importing data. The visualization interface displays the position and information of the disease on the two-dimensional map, which can be switched to the roughness display mode at the same time.