In this paper, the algorithm which improve the efficiency of surveillance in spite of the change of light is proposed and confirmed by virtue of the experiments. One of the problems for the implementation of visual surveillance system is the image processing technique to overcome with the variations of illumination lights. Some conventional systems are generally not considered the error due to the change of lights because the system use at indoor. In practical, the factors of bad image can be classified to the ghosts due to the reflection of lights and shadows in a scene. Especially weak images and noises at night are decreased the performance of visual surveillance system. In the paper, the filter which improve the images with some change of illumination lights is designed and the gabor filter is used for recognition and tracking of the moving objects. In the results, the system showed that the recognition and tracking were obtained $92sim100%$ of recognition rate at daytime, but $80sim90%$ of nighttime.