This paper propose a new frame level rate control algorithm for improving video quality and decreasing quality variation of an entire video sequence in a very low bit rate environment. In the proposed scheme, the allocated bits to a GOP are distributed to each frame properly according to the frame characteristics as well as the buffer status and the channel bandwidth. The H.264 standard uses various coding modes and optimization methods to improve the compression performance, which makes it difficult to control the generated traffic accurately. In this paper, proper prediction models for low bit rate environments are lust proposed, and a target distortion is determined using the models. According to the target distortion, the bit budget is allocated to each frame. It is shown by experimental results that the new algorithm can generate the PSNR performance better than that of the existing rate control algorithm.