Deep Learning Detection Algorithm for Surface Defects of Automobile Door Seals

: The surface defects of automobile door seals are mainly detected manually at present, which is costly and has low efficiency. Therefore, a deep learning automatic detection algorithm of automobile door seals is studied in this paper. At first, the defects are classified and the data set is made according to the geometric characteristics of the defects. While enhancing the data set, the K-means clustering algorithm is used to cluster the target annotation frame in the data set, and the anchor that matches with the surface defect size of the seal is obtained. Finally, in view of the characteristics of large variation range of defect size, the target detection algorithm YOLOV3 is selected as the basic framework. Meanwhile, considering the high proportion of small and medium-sized targets in defects, the output scale is introduced at 4 times of the sampling position of YOLOV3 backbone network. In order to further enhance the correlation between the extracted features and the channel, spatial position and coordinate position, the feature fusion and attention mechanism module is constructed. The test results show that the mean of the average precision is improved by 4.81% compared with YOLOV3.


INTRODUCTION
Product surface defects seriously affect the beauty, sales and price of products.It is of great practical significance to find surface defects as soon as possible in the production process.In recent years, the automation of the automobile industry has developed rapidly, but the surface defects detection technology of automobile door seals obviously lags behind and still depends on manual detection, which has the disadvantages of low efficiency, high detection cost and human subjective judgment.
At present, surface defects detection methods mainly include traditional image processing methods and deep learning methods [1][2][3].Traditional image processing methods detect targets through edge detection, threshold segmentation, feature histogram, classical machine learning methods [2,[4][5][6][7][8][9][10] (support vector machine, k-Nearest Neighbor method and Naive Bayes, neural network, decision tree, etc.).Literature [11] constructed neural network to detect welding defects under alternating/rotating magnetic field, and the test detection accuracy is 94.1%.Literature [12] detected paint defects on car bodies through image preprocessing, threshold segmentation and other operations.The experimental results show that under the condition of static image acquisition, the average detection rate of defects is greater than 95%.Although traditional methods have been widely used in production and life, they are still vulnerable to the influence of environment, illumination, production process and noise [1].At the same time, it is difficult to build a stable, reliable and robust algorithm for the defects of a wide variety, diverse forms and small image contrast.Compared with traditional image processing methods, deep learning methods [13,14] have greatly improved in detection speed, accuracy and generalization ability, and are widely used in the field of target detection [15][16][17][18].More and more scholars apply deep learning to the field of defects detection [1].For example, the improved YOLOV3 is used to detect the surface defects of commutator.The implementation results show that the detection speed and accuracy are better than YOLOV3 when the model parameters are reduced [19].Reference [20] proposed a deep convolution neural network to automatically find the optimal parameters of convolution kernel to detect welding defects, with an average detection accuracy of 93.36%.Reference [21] detected rolled metal surface defects based on resnet50 neural network, and the detection accuracy is as high as 96.91%.Literature [22] introduced the attention mechanism into YOLOV3 to detect the surface defects of automobile door seals.Experiments show that the mean of the average precision of the improved YOLOV3 algorithm is 3.82% higher than that of the original YOLOV3.
Although deep learning has been widely used in defects detection, few scholars apply deep learning to detect the defects of automobile door seals.This paper studies the surface defects detection method of automobile door seals based on deep learning.Firstly, a certain number of seal surface defects images are collected, and the defects are classified and marked according to the geometric characteristics and shape features of the defects.Then, an improved YOLOV3 network is designed according to the characteristics of seal surface defects.The experimental results show that the improved algorithm has a greater improvement than YOLOV3 average detection precison when the detection speed is with little to no variation.

DATA SET PRODUCTION AND ANALYSIS
There are various forms of surface defects of automobile door seals.According to the geometric characteristics and shape features of defects, defects are divided into 8 categories: pit, hole, wrinkle, bulge, crackle, scar, swelling and scratch.The types and characteristics of seal defects are shown in Tab. 1. Fig. 1 is the physical diagram of different types of defects on the surface of seals.A total of 3796 images were collected, and the size of the image was 1024  768 pixels.The defects were marked and a data set was made.The statistics of the number of various defects in the data set are shown in Tab. 2. In order to design an algorithm for detecting surface defects of automobile door seals, the characteristics of defects in the data set are analyzed first.It can be seen from Fig. 2 that defects with rectangular frame area less than 1% of image area marked with seal surface defects account for 38% in the data set, that is, the proportion of small targets [23] in the data set is greater than 38%.Defects whose rectangular frame area is larger than 10% of the image area account for less than 2% in the data set, that is, there are few targets with large size.It can be seen from Fig. 3 that the area range of rectangular frame marked with different types of defects and the same type of defects is large, that is, the area difference of different types of defects is large, and the size difference of similar defects is large.Among them, the area difference of rectangular frame marked with crackle, scratch and scar defects is more significant.The rectangular frame area marked with swelling, bulge, pit, is mostly more than 8000 pixels, that is, the defects belonging to small targets account for a high proportion of these defects.As can be seen from Fig. 4, the number of defects whose size along the image width W direction of the rectangular frame marked with defects is less than 10% of the image size, account for 44% of the total number of defects.The number of defects whose size along the image height H direction of the rectangular frame marked with defects is less than 10% of the image size accounts for 38% of the total number of defects.That is, the size data of the rectangular box marked with defects in the H direction has a high proportion of small targets.As can be seen from Fig. 5, the width height ratio of rectangular frame marked with similar defects varies widely, among which the width height ratio of rectangular frame marked with crackle and scratch defects is less stable, and the width height ratio of rectangular frame marked with different types of defects is mostly concentrated near the value 1.
To sum up, (1) the defect area in the data set changes greatly, the proportion of defects belonging to small targets is relatively high, and the proportion of defects belonging to small targets in swelling, bulge and pit defects is higher.
(2) The proportion that the size of the rectangular box marked with defects in the dataset is less than 10% of the image size is higher than the proportion that the defects in the dataset belong to small targets.(3) The width height ratio of most rectangular frames marked with defects is relatively stable.( 4) The aspect ratio of some types of defects changes greatly, especially the scratch and crackle defects.

SURFACE DEFECT DETECTION NETWORK OF AUTOMOBILE DOOR SEAL 3.1 YOLOVNetwork Optimization and Improvement
Fig. 6 shows the YOLOV3 network model.Its main feature extraction network is darknet53, which includes 53 convolution operations, batch normalization and leaky relu activation functions.In the figure, there are 5 down samples from bottom to top, and the feature pyramid enhancement features are constructed by using the feature maps of 8 times, 16 times and 32 times down sampling, which are output at three scales.The proportion of seal surface defects belonging to small targets is high, the marked size of some defects is less than 5% of the image, and the marked frame width and height of defects such as scratches and crackles are relatively large.Defects with large width height ratio, defects belonging to small targets and defects less than 5% of the image size contain less defect information in one or two dimensions.During the 8 times down sampling process, the characteristic information of some defects has been seriously lost, and even the characteristic information of some defects has completely disappeared, especially defects with large width height ratio such as scratches and crackles.In order to avoid the loss of defect feature information, an output scale is added in the sampling layer under 4 times of the backbone network.The setting of anchor size has an important impact on the final detection results [24].After determining the output scale, K-means algorithm is used to cluster the defect marking frame to obtain anchor matching the target size.The results are: (15,20), (24,29) The surface defects image of automobile door seal is transmitted to darknet53 network for preliminary defect feature extraction, and then four output scale feature pyramids are constructed using 4, 8, 16 and 32 times down sampling feature maps, namely (104, 104), (52, 52), (26,26), (13,13) to enhance the features extracted by the backbone network.Specific process: 32 times down sampling feature maps 0 (1024  13  13), maps 1 (256  26  26) is obtained by 6 convolutions and one upsampling.After 7 convolution operations, the channel is transformed into 39, and the size is 13 × 13.The feature maps 1 is concatenated with the 16 times down sampling feature maps in the backbone network to obtain the feature maps 2 (768  26  26).The concatenated feature maps 2 is convoluted six times and upsampling once to obtain the feature maps 3 (128  52  52).After 7 convolution operations, the channel is transformed into 39, and the size is 26  26.The feature maps 3 is concatenated with the feature maps 4 of 8 times down sampling in the backbone network to obtain the feature maps 5 (384  52  52).The concatenated feature maps 5 is convoluted six times and upsampling once to obtain the feature maps 6 (64  104  104), which is transformed into 39 channels  In order to further enhance the extracted features, the four scale feature maps are fused, and the attention mechanism is introduced to obtain the detection results of surface defects of automobile door seals through the output network.The improved YOLOV3 network structure is shown in Fig. 7.

Feature Fusion and Attention Mechanism Network
As shown in Fig. 8, after the image of automobile door seal passes through the backbone network darknet53 and the feature pyramid network (FPN), the feature maps 1, feature maps 2, feature maps 3 and feature maps 4 are obtained.The four feature maps are fused after upsampling or down sampling.The specific process is shown in Fig. 4, where US (s = 2) represents the upsampling with scale factor of 2, the previous multiple represents the number of up sampling, DC (s = 2) represents the dilated convolution with dilated coefficient of 2, and the previous coefficient represents the number of hole convolutions.Here, the dilated convolution is used for down sampling.The detailed process of feature fusion is as follows: for scale 1, the shallow high-resolution feature maps 2, feature maps 3 and feature maps 4 are convoluted once, twice and three times with a dilated coefficient of 2. For scale 2, the upsampling with scale factor of 2 is used to upsample the deep-seated low-resolution feature maps 1, and the shallow high-resolution feature maps 3 and feature maps 4 are respectively subjected to cavity convolution with cavity coefficient of 2 once and twice.For scale 3, the deep-seated low-resolution feature maps 1 and feature maps 2 are up sampled twice and once with a scale factor of 2 respectively, and the shallow high-resolution feature maps 4 is convoluted once with a hole convolution with a hole rate of 2. For scale 4, feature maps 1, feature maps 2 and feature maps 3 are upsampled 3 times, 2 times and 1 time respectively by upsampling with scale factor 2. Finally, the feature maps of the same size are concatenated by series operation, and the importance and correlation between channel, spatial position, position coordinates and features are learned through attention mechanism, and the detection results are output.In order to select the optimal attention mechanism, squeeze and excitation (SE) attention mechanism [25], convolutional block attention module (CBAM) [26] and coordinated attention mechanism [27] are used for comparison.

TEST PROCESS AND RESULT ANALYSIS 4.1 Experimental Parameters
The parameters of the test system are as follows: the deep learning framework is pytorch, version 1.2 0, python version is 3.7.Using Adam optimization algorithm, the initial value of learning rate is set to 0.001 and the confidence is 0.5.In order to ensure that the model loss converges to a stable value, the epoch of the test is set to 100.

Quantitative Analysis
For the convenience of description, the names of the test network are numbered as 1-5.It can be seen from Tab. 3 that the average detection accuracy of the surface defect detection network of automobile door seals is 55.12% by using the original YOLOV3 network.Considering the small size of automobile door seal defects, after adding an output scale in the feature pyramid network, the average precision (AP) values of all types of defects except pit defects are increased, in which the AP values of crackle and scratch defects are increased by 2.64%, 6.1% respectively and mAP increased 3.43%.It shows that increasing the output scale in the shallow layer can effectively reduce the problem of information loss when sampling under large length width ratio defects.
By introducing feature fusion and SE module, the detection accuracy AP of swelling defects is 14.72% higher than that of the original YOLOV3 network and 3.2% higher than that of network 2. The detection AP value of wrinkle defects is 6.37% higher than that of network 1, 5.67% higher than that of network 2, and the mean average precision (mAP) is 3.74% higher than that of original YOLOV3 and 0.31% higher than that of network 2, indicating that the introduction of feature fusion and SE module can enhance the defect features extracted by the network.
The feature fusion and CBAM module are introduced.Compared with the feature fusion with SE attention mechanism, the detection accuracy of all defects except pit and scratch defects is improved, and the map is improved by 1.07%.Among them, the detection accuracy of swelling defects was improved by 3.04%, hole defects by 3.43% and wrinkle defects by 2.89%.Compared with network 1, mAP is improved by 4.81%, in which the detection accuracy of bulge defects is improved by 17.76%, wrinkle defects by 9.26%, scratch defects by 3.64% and bulges defects by 3.85%, indicating that the introduced feature fusion and CBAM module can effectively enhance features.
By introducing the feature fusion and coordinate module, compared with network 2, the mAP is reduced by 3.42%, and the detection accuracy of all types of defects except crackle defects is reduced.Compared with model 1, the detection accuracy is almost unchanged.Compared with network 3, the mAP decreases by 3.73%.Further analysis shows that the seal image size of the output network is resized to 416 × 416, after 32 times down sampling, the small target is only one pixel in the 13 × 13 feature maps.When the feature fusion and coordinate module is introduced, the relationship between unconnected coordinates is enhanced, resulting in a significant decline in the detection results.
It can be seen from Tab. 3 and Tab. 4 that the feature pyramid network is constructed by using the down sampling feature maps of 4 times, 8 times, 16 times and 32 times of the backbone network, and the features are further enhanced by feature fusion and CBAM module.The decrease of detection speed is negligible, and the mAP is 4.81% higher than the original YOLOV3.

Qualitative Analysis
Fig. 9 is the difference curve of defect AP detected by different networks.Fig. 10 is a partial P-R curve of defect detected by network 1 and network 4. It can be seen from Fig. 9a that the detection effects of the other three detection models are improved compared with the original network except the fifth network model.It can be seen from Fig. 9b that the detection effect of network 4 is the best.When the AP of some defects decreases slightly, the detection accuracy of scratch, bulge and wrinkle is significantly improved.That is, it can be seen from Fig. 11 that the P-R curve area of defects detected by network 4 is greater than that of defects detected by network 1. Fig. 11 is a visual result of the detection of surface defects of automobile door seals on five convolutional neural networks.As can be seen from the figure, when network 1 is used for detection, there is a high missed detection rate.Hole defects can be detected after increasing the output scale, but crackle and pit defects have missed detection.When model 4 is used for detection, the missed detection rate is significantly reduced.
From the quantitative analysis, the AP and mAP parameters of network 4 are better than other networks.From the qualitative analysis, the stability of network 4, the area of defect P-R curve and the visualization effect of curve detection are better than other networks, indicating that the YOLOV3 network improved for the defects of automobile door seals has a good detection effect.

CONCLUSIONS
In view of the problem of surface defects detection of automobile door seals, an automobile door seal detection method based on deep learning is studied.In this paper, the defect images of automobile door seals are collected, and the defects are classified and marked according to the geometric characteristics, geometric dimensions and other parameters of the defects.By analyzing the data set of surface defects of seals, the YOLOV3 model is improved according to the characteristics of defects.In order to improve the positioning accuracy of target detection, Kmeans algorithm is used to optimize anchor.Through the comparative test, it is found that an output scale is added at the sampling point 4 times lower than the backbone network, and the mAP is improved by 3.43%.After the feature pyramid, feature fusion and CBAM attention mechanism are introduced.The mAP is 59.93%, which is 1.07% higher than that of network 2. Compared with the original YOLOV3, the AP of crackle defects with large length width ratio is improved by 3.64%, scratch defects by 7.87%, swelling defects with no obvious characteristics by 17.76% and mAP by 4.81%.

Figure 2
Figure 2 Statistics on the ratio of rectangular frame area to image area for marking surface defects of seals

Figure 3
Figure 3 Statistics of the area of the rectangular frame for marking the surface defects of the seal

Figure 4 Figure 5
Figure 4 Statistics on the ratio of the length and width of the labelling frame to the surface defect of the seal and the ratio of the length and width of the image
of defects to total number(%) Range of dimension box size to image size ratio(%)W direction H direction after 7 times of convolution operation, and the size is 52  52 feature maps 3. The feature maps 6 is concatenated with the four times down sampled feature maps in the backbone network to obtain the feature maps 7 (192  104  104).The concatenated feature maps 7 is transformed into 39 channels by 7 convolution operations, and the size is 104 × 104.

Figure 7
Figure 7 Network structure of Improved YOLOV3 automobile door seals detection model

Figure 8
Figure 8 Feature fusion module based on attention mechanism

Figure 9 Figure 10 P
Figure 9 Comparison chart of AP difference detected by different models

Figure 11
Figure 11 Effect of different network models on detecting automobile door seals

Table 1
Classification and characteristic description of surface defects of automobile door seals

Table 2
Statistics on the number of various defects in the surface defect data set of automobile door seals

Table 3
Comparison of experimental results of different algorithms for defect detection

Table 4
Speed comparison of different algorithms for defect detection