In this paper, a fire detection algorithm based on video processing is proposed. At the first stage, background image extracted from CCTV video input signal, and then foreground image were separated by differencing CCTV input signal from background image. At the second stage, candidated area were extracted by using color information from foreground image. At the final stage, smoke or flame characteristic area were separated by using Gaussian mixture modeling applied to candidated area, and then fire can be detected. Through real experiments at the inner room, it is shown that the proposed system works well.