In this paper, the visual tracking system for arbitrary shaped moving object is proposed. The established tracking system can be divided into model based method that needs previous model for target object and image based method that uses image feature. In the model based method, the reliable tracking is possible, but simplification of the shape is necessary and the application is restricted to definite target mod el. On the other hand, in the image based method, the process speed can be increased, but the shape information is lost and the tracking system is sensitive to image noise. The proposed tracking system is composed of the extraction process that recognizes the existence of moving object and tracking process that extracts dynamic characteristics and shape information of the target objects. Specially, active contour model is used to effectively track the object that is undergoing shape change. In initializatio n process of the contour model, the semi-automatic operation can be avoided and the convergence speed of the contour can be increased by the proposed effective initialization method. Also, for the efficient solution of the correspondence problem in multiple objects tracking, the variation function that uses the variation of position structure in image frame and snake energy level is proposed. In order to verify the validity and effectiveness of the proposed tracking system, real time tracking experiment for multiple moving objects is implemented.