For the object-based video compression at very low bit rate, vieo segmentation is an essential part. In this paper, we propose a temporal video segmentation algorithms for motion pictures which is based on blocks. The algorithm is composed of three steps: (1) the change-detection, (2) the block merging, and (3) the block segmentation. The first step separates the change-detected region from background. Here, a new method for removing the uncovered region without motion estimation is presented. The second step, which is further divided into three substeps, estimates motions for the change-detected region and merges blocks with similar motions. The merging conditions for each substep as criteria are also given. The final step, the block segmentation, segments the boundary block that is excluded from the second step on a pixel basis. After describing our algorithm in detail, several experimental results along the processing order are shown step by step. The results demonstrate that the proposed algorithm removes the uncovered region effectively and produced objects that are segmented well.