Abstract: This study addressed the design of a home robot based on the visual navigation control system. In order to accomplish basic home services, the robot is designed to have the functions of environmental map construction, autonomous navigation, collision-free control and human-robot interaction. For environmental sensing and object tracking, a series of image features recognition procedures are used to establish the database of objective images. By integrating image information of objectives and ground lines, the robot can determine the relative localities of specific objectives and then perform environmental map construction and robot localization. Moreover, the user can also control the robot remotely to navigate and construct the environmental map through the interaction website. The experimental results demonstrate the environmental adaptability of the robot based on the proposed visual navigation control system.