Many automobile manufacturers are doing experiment on manufacturing environments by using an augmented reality technology. However, system layout and process simulation by using the virtual reality technology have been performed actively more than by using the augmented reality technology in practical use so far. Existing automobile assembly by using the augmented reality requires the precise calibrating work after setting the robot because the existing augmented reality system for the automobile assembly system configuration does not include the end tip deflection and the robot joints deflection due to the heavy weight of product and gripper. Because the robot is used mostly at the automobile assembly, the deflection problem of the robot joint and the product in the existing augmented reality system need to be improved. Moreover camera lens calibration has to be performed precisely to use augmented reality. In order to improve this problem, this paper introduces a method of the software based calibration to apply the augmented reality effectively to the automobile assembly system. On the other hand, the camera lens calibration module and the direct compensation module of the virtual object displacement for the augmented reality were designed and implemented. Furthermore, the developed automobile assembly system oriented AR-system was verified by the practical test.