This paper proposes a method of localization of vehicle especially the horizontal position for the purpose of recognizing the driving lane. Through tracking road signs, the relative position between the vehicle and the sign is calculated and the absolute position is obtained using the known information from the regulation for installation. The proposed method uses Kalman filter for road sign tracking and analyzes the motion using the pinhole camera model. In order to classify the road sign, ORB(Oriented fast and Rotated BRIEF) features from the input image and DB are matched. From the absolute position of the vehicle, the driving lane is recognized. The Experiments are performed on videos from the highway driving and the results shows that the proposed method is able to compensate the common GPS localization errors.