In order to make the effect of the navigation system more direct, the paper proposes a thought of vehicle navigation system based on Video-GIS. A semantic framework has been defined whose core is focused on the integration and interaction of video and spatial information, which supports full content retrieval based on multimodal metadata extraction and fusion, and supports kinds of wireless access mode. Furthermore, requirements of prototype system are discussed. Then the design and implementation of framework are discussed. Next, describe the key ideas and technologies involved. Finally, we point out its future research trend.