Multi-Target Detection and Tracking from a Single Camera in Unmanned Aerial Vehicles (UAVs)



Dataset

Data Set: The data set comprises 50 video sequences of 70250 frames with 30 fps frame rate. They are recorded by a GoPro 3 camera (HD resolution: 1920080 or 128060) mounted on a custom delta-wing airframe. For each video, there are multiple target UAVs (up to 8) which have various appearances and shapes. We manually annotate the targets in the videos by using VATIC software to generate ground-truth dataset for performance evaluation.
Videos - A dataset of 50 videos taken from real UAVs with one or multiple UAVs in the view.
Annotations - Annotation of the videos.

Algorithm

We present a new approach to detect and track other UAVs from a single camera on our own UAV. Given the sequence of video frames, we estimate background motions via perspective transformation model and then identify distinctive points in the background subtracted image to detect moving objects. We find spatio-temporal characteristics of each moving object through optical flow matching and then classify our targets which have very different motion compared with background. We also perform Kalman filter tracking to enforce the temporal consistency on our detections. The algorithm is tested on real videos from UAVs and results show that our algorithm is effective to detect and track small UAVs with limited computing resources.

no im

Challenge in detecting other UAVs: [Left] Original Video, [Right] Video with our detection and tracking; Other UAVs are very small and occluded by complex backgrounds (i.e. cloud) and thus not even recognizable by human eyes. Our proposed method detects and tracks multiple small UAVs successfully as highlighted in red boxes.

References
paper - Jing Li, Dong Hye Ye, Timothy Chung, Mathias Kolsch, Juan Wachs, Charles Bouman, "Multi-target detection and tracking from a single camera in Unmanned Aerial Vehicles (UAVs)." Intelligent Robots and Systems (IROS), 2016.