Ground target localization
The aim of this project is to provide the accurate location of ground objects in a world coordinate system using vision-enabled unmanned aerial vehicles (UAVs). Our approach pursues to find the correct registration between aerial images and reference image. In other words we try to solve the homography relating the image plane in the camera coordinate system to the ground plane in the world coordinate system instead of using common backprojection. For computing the homography between the image and world plane, we create a set of correspondences between two planes by identifying common known landmarks, such as buildings, roads, bridges, etc. We assume that the geo-location of these man-made landmarks is known by referencing geographic information system (GIS). Therefore the goal of this approach focuses on how to extract candidate landmarks from aerial images and to match them to reference landmarks.
Our approach is done through a three step processes: (1) extract lines from the aerial images, (2) estimate an initial homography, and (3) match the line to a set of reference landmarks. Fig. 1 illustrates the whole system structure of ground target localization.
Figure 1: The system structure of ground target localization
In process (1), we extract lines from each aerial image because lines can capture the shape of man-made landmarks efficiently. To remove artifacts of extracted lines occurring in natural regions, we develop new line extraction algorithm that can remove small fragments of lines or merge them into one line.
After extracting lines from the aerial image, we convert them from the camera coordinate system to world coordinate system through initial camera-to-world homography. The first method computes the initial camera-to-world homography from precomputed camera calibration and GPS/IMU information. The second method for the initial camera-to-world homography is to use the frame-to-frame homography, i.e., the homography from the previous frame to the current frame. Fig. 2 explains the relationship among the camera-to-world homography of the previous frame as well as the current frame and frame-to-frame homography.
Figure 2: Illustration of the relationship among the camera-to-world homography of the previous frame as well as the current frame and frame-to-frame homography.
In process (3), we can estimate the refined camera-to-world homography based on results of process (2). The refined homography can be estimated by a chain of the current camera-to-world homography and a rigid transformation in the world plane. The rigid transformation is computed by finding the correspondent lines between aerial images and the reference image through Iterative Closest Point(ICP) algorithm. We develop a point-to-point ICP with comparing normal vector of each point so that we can achieve line-to-line matching.
Figure 3: The example of registering aerial images and reference image by ICP
Fig.3 shows the results of our ground target localization approach. Red lines and blue lines are extracted from a satellite image (reference image) and from an aerial image, respectively. The green lines represent the correspondences between matched points from two line sets. The yellow, cyan, and white circle indicate the ground-truth object location, estimated object location by our approach, and estimated location by backprojection, respectively. As shown in Fig. 3, our approach shows significant improvements of target localization.
• Kyuseo Han
• Chad Aeschliman
• Johnny Park
Sample Video [mpg , 5.5 MB]
The final registered aerial images are overlaid into the reference image. Some frame in the middle of aerial image sequences cannot be correctly registered due to abrupt viewpoint changes. However, our registration can be quickly stabilized again. The top sequence shows the results of backprojection with only GPS information and the bottom sequence displays the resulst of our approach.
K. Han, C. Aeschliman, J. Park, A. C. Kak, H. Kwon, and D. Pack, "Accurate Ground Target Localization based on Landmark Registration for Unmanned Aerial Vehicles," submitted to IROS 2011.
based on Landmark registration in aerial images