AVENUE: Visual Localization Results

To perform the visual localization experiment, we created an accurate model database of various buildings throughout Columbia University Morningside campus (Fig.1). The model was created by scanning prominent features with an electronic theodolite (a.k.a. total station). The features modeled were windows, ledges and linear decorations --- all commonly found and abundant in urban landscapes.

To test the accuracy of the visual localization method, a total of 16 tests were performed at various locations throughout the campus. The robot was driven manually along a long trajectory, stopped at each of the test locations and directed to perform the visual localization routine. It used the odometry pose estimates as an initial guess to determine the nearby buildings and choose which model to use.

Figure 2 in the side panel shows a 2-D map of the campus. The locations of the tests are marked with black dots and the orientations of the cameras at these locations --- with arrows. The yellow and red colors outline buildings and the green color outlines vegetation.

Figure 3 is a table that illustrates visually the results for each of the test locations. Each row corresponds to the test at the location number shown in the first column. The left image in the row illustrates the initial pose guess by overlaying the model of the building on the image taken by the camera. The discrepancy between the projection of the model and the image illustrates the innaccuracy of the initial guess. The image to the right shows the same model projected onto the same image after the visual pose estimation step. You can see that in all cases the alignment is very good. The last column shows the error of the position estimate compared to ground truth data.

The goal of these experiments was to verify that the visual localization algorighm produces consistent results. Two such experiments were performed in two different locations. In each of these locations, a pair of images were taken of different building facades by only turning the camera without changing its position significantly. The visual localization algorithm was performed on both images and the position results were compared.

The figures linked to from the side panel illustrate the results visually in the form of a table. The first row of the table is related to the first image of the pair and the second table row refers to the second image. The first column shows the image with the corresponding model overlaid using the initial pose estimate (i.e. the before image), while the second column illustrates the computed pose of the camera starting from the initial guess (i.e. the after image). For consistency experiment 1, the estimate inconsistency between the two images was 0.064m (Fig.4). For consistency experiment 2, the inconsistency was 0.290m (Fig.5).

Finally, an experiment was performed to confirm that the entire robot localization system works well together, i.e. it uses the visual localization method as needed and that it actually improves the performance. A more than 330m long trajectory was composed on the campus. The robot was directed to follow autonomously the trajectory using all sensors and choosing the localization method as needed.

During the test run, the robot passed through both areas of good GPS coverage and poor GPS coverage (Fig.6). It was able to consistently detect the areas of poor GPS performance (marked in the figure) and employ the visual method to improve its pose estimation accuracy. Notice that no GPS data was available at all at location 3, as the robot was directly beneath an extension of the nearby building.

The table below shows the position errors of the open space localization method and the visual localization method at each of these locations. It clearly illustrates the improvement the visual localization method brings.

The experiments above show that accurate mobile robot localization in urban environment is possible. The first set of experiments shows an average error of 0.261m, which is definitely acceptable for the task at hand. The second set of experiments confirms that the results are consistent to a high degree regardless of the building chosen to perform the visual experiment on. Finally, the last experiment verified in practice the feasibility of autonomous navigation in urban terrain using the combination of open-space and visual localization methods as well as the improved performance of the combined system over using only GPS and inertial sensors.