The stability of object tracking in non-stationary camera environment, such as intelligent surveillance system using a pan/tilt camera, is less stable compared with stationary camera environment. This is due to the fact that it is difficult to model a background image in non-stationary environment. In this letter, we propose a non-stationay pan/tilt camera surveillance system which uses a stop/go procedure together with a real-time active contour. The proposed system can track the object stable even in an environment where only a few difference frames can be obtained.