Multi-Target Detection and Tracking from a Single Camera in Unmanned Aerial Vehicles (UAVs)


Data Set: The data set comprises 50 video sequences of 70250 frames with 30 fps frame rate. They are recorded by a GoPro 3 camera (HD resolution: 1920x1080 or 1280x1060) mounted on a custom delta-wing airframe. For each video, there are multiple target UAVs (up to 8) which have various appearances and shapes. We manually annotate the targets in the videos by using VATIC software to generate ground-truth dataset for performance evaluation.
Videos - A dataset of 50 videos taken from real UAVs with one or multiple UAVs in the view.
Annotations - Annotation of the videos.


GitHub site with code - We present a new approach to detect and track other UAVs from a single camera on our own UAV. Given the sequence of video frames, we estimate background motions via perspective transformation model and then identify distinctive points in the background subtracted image to detect moving objects. We find spatio-temporal characteristics of each moving object through optical flow matching and then classify our targets which have very different motion compared with background. We also perform Kalman filter tracking to enforce the temporal consistency on our detections. The algorithm is tested on real videos from UAVs and results show that our algorithm is effective to detect and track small UAVs with limited computing resources.

no im

Challenge in detecting other UAVs: [Left] Original Video, [Right] Video with our detection and tracking; Other UAVs are very small and occluded by complex backgrounds (i.e. cloud) and thus not even recognizable by human eyes. Our proposed method detects and tracks multiple small UAVs successfully as highlighted in red boxes.

paper - Jing Li, Dong Hye Ye, Timothy Chung, Mathias Kolsch, Juan Wachs, and Charles A. Bouman, ``Fast and Robust UAV to UAV Detection and Tracking from Video,'' to appear in {\em Transactions on Emerging Topics in Computing}. doi:10.1109/TETC.2021.3104555
paper - Jing Li, Dong Hye Ye, Timothy Chung, Mathias Kolsch, Juan Wachs, Charles Bouman, "Multi-target detection and tracking from a single camera in Unmanned Aerial Vehicles (UAVs)." Intelligent Robots and Systems (IROS), 2016.
paper - Dong Hye Ye, Qiulin Chen, Jing Li, Juan Wachs, and Charles Bouman, ``Deep Learning for Moving Object Detection from a Single Camera in UAVs,'' {\em Imaging and Multimedia Analytics in a Web and Mobile World 2018} at {\em IS\&T International Symposium on Electronic Imaging 2018}, Feb. 2016.