This paper presents compression of image sequences based on the classification of interframe difference image blocks. classification process consists of image activity classification and energy distribution classification. In the activity classification, interframe difference image blocks are classified into activity blocks and non-activity blocks using the edge detection. In the distribution classification, activity blocks are further classified into vertical blocks, horizontal blocks, and small activity blocks using the AC energy distribution features. The RBFN, trained with numerical classification results, successfully classifies difference image blocks according to image details. Image sequence compressing based on the classification of interframe difference image blocks using the RBFN shows better compression results and less training time than the classical sorting method and the MLP network.