In this paper, a new scheme to recognize a finger shape in the depth image captured by Kinect is proposed. Rigid transformation of an input finger shape is pre-processed for its robustness against the shape angle of input fingers. After extracting contour map from hand region, observing the change of contour pixel location is performed to calculate rotational compensation angle. For the finger shape recognition, we first acquire three pixel points, the most left, right, and top located pixel points. In the proposed algorithm, we first acquire three pixel points, the most left, right, and top located pixel points for the finger shape recognition, also we use geometrical features of human fingers such as Euclidean distance, the angle of the finger and the pixel area of hand region between each pixel points to recognize the finger shape. Through experimental results, we show that the proposed algorithm performs better than old schemes.