Nowadays, a private protection act has come into effect which demands for the protection of personal image information obtained by the CCTV. According to this act, the object out of interest has to be mosaicked such that it can not be identified before the image is sent to the investigation office. Meanwhile, the demand for digital videos obtained by CCTV is also increasing for digital forensic. Therefore, due to the two conflicting demands, the demand for a solution which can automatically mask an object in the CCTV video is increasing and related IT industry is expected to grow. The core technology in developing a target masking solution is the object tracking technique. In this paper, we propose an object tracking technique which suits for the application of CCTV video object masking as a postprocess. The proposed method simultaneously uses the motion and the color information to produce a stable tracking result. Furthermore, the proposed method is based on the centroid shifting method, which is a fast color based tracking method, and thus the overall tracking becomes fast.