In this paper, we proposed an algorithm implementing mouse functions using hand motion and number of fingers which are extracted from an image sequence. The sequence is acquired through a web camera and processed with image processing algorithms. The sequence is first converted from RGB model to YCbCr model to efficiently extract skin area and the extracted area is further processed using labeling, opening, and closing operations to decide the center of a hand. Based on the center position, the number of fingers is decided, which serves as the information to decide and perform a mouse function. Experimental results show that 94.0% of pointer moves and 96.0% of finger extractions are successful, which opens the possibility of further development for a commercial product.