Vision-based Aircraft Detection

  Recently, recognizing that a highly reliable collision avoidance capability equivalent to that of a human pilot is necessary to fulfill the mission in civilian airspace, unmanned aerial vehicles with a total weight of 150kg or more are in the process of establishing relevant regulations at ICAO, FAA and EASA. UAVs can meet the safety requirement utilizing cooperative collision avoidance systems such as TCAS or ADS-B, but since small UAVs are not able to mount that equipment and have weight and power considerations, non-cooperative vision-based aircraft detection techniques using a single camera have been studied. Based on saliency detection by analyzing the changes in the image characteristics as frame continues, Track- before-detect approach was mainly performed with dynamic programming, Hidden Markov Model, and particle filter method. In case of using multiple frames, aircraft are detected through the feature vectors’ continuous change rate using homography matrix based background subtraction and particle filters. With these image processing methods, aircraft detection in sky and ground region has been demonstrated. Air- craft detection using single camera by spatio-temporal cube method with machine learning approach has been also demonstrated. The image processing methods mentioned above show high performance in se- lected environment settings, but may not work well and need to tune threshold or parameters in certain environment. One example is that detection method using image subtraction may struggle in situation that the target aircraft displays little relative motion to the background. In addition, since the distant target and the near target have large difference in the signal-to-noise ratio (SNR) characteristics, different algorithms may need to obtain proper performance. To overcome these shortages, we propose a vision-based detection method by deep learning approach. Recently, computation power and speed of GPU has been dramatically increased, so that processing large amount of computation is enabled even with a small on-board computer and this makes deep learning could be applied for small UAS.

  Considering our constraints, we changed the SSD architecture by converting, removing, and adding some layers and modified several settings. Our base network is based on AlexNet. Since we want more accurate classification, a pair of conv and pooling layer was added after conv5. Then we converted two fully connected layers(fc6, fc7) into conv layers and removed the last one(fc8) to fit SSD structure. The last two extra feature layers are removed because we rarely need to detect full-screen-sized objects. Box prediction sampling layers are changed as well for better performance on small objects. Compared to the original SSD, our model samples more from the base network. We also adjusted default box sizes of the model for further improvement. Since the target aircraft mainly appears as a small number of pixels, we use small default box sizes starting from 10 pixels and ended in 214 pixels. On our 17k frames test data, the improvement in detection success rate is 15.8% compared to the original VGG-16 based SSD setting. Our network model achieved an average inference time of 38.3ms, which is about 26.1fps. It is fast enough for real-time processing and roughly six times faster than using VGG-16 as base network.