Abstract: Robust hierarchical method is studied for implementing indoor mobile robot localization and autonomous navigation with omnidirectional vision. The localization is purely vision-based and uses a sequence of prerecorded image frames as reference topological map. For finding the image which best fits the current image, the system consists of two hierarchical components: global coarse localization and local accurate localization. The global color histograms are used to confirm fast the coarse robot position, which is not accurate, whereas the more accurate position can be obtained slowly through local scale invariant interest keypoints, which can be robustly matched with nearest neighbor search based on KD-Tree. The navigation system is composed of on-line and off-line stages. During the off-line learning stage, the robot performs paths and records a set of ordered key images. From this sequence a topological map is built. Each topological node provides a set of omnidirectional images characterized by geometrical scale invariant keypoints. In the on-line navigation stage, the robot recognizes the most likely node through robust hierarchical algorithm and then controlled by a vision based control law adapted to omnidirectional cameras to follow the visual path. To evaluate this method, actual experiment is carried out in an indoor environment. Results show the robustness and efficiency of the proposed method.