Non-rigid registration between different modality images with shape deformation can be used to diagnosis and study for inter-patient image registration, longitudinal intra-patient registration, and registration between a patient image and an atlas image. This paper proposes a hierarchical registration method using bodily tissue based segmentation for registration between color images and CT images of the Visible Human leg areas. The cross-sectional color images and the axial CT images are segmented into three distinctive bodily tissue regions, respectively: fat, muscle, and bone. Each region is separately registered hierarchically. Bounding boxes containing bodily tissue regions in different modalities are initially registered. Then, boundaries of the regions are globally registered within range of searching space. Local boundary segments of the regions are further registered for non-rigid registration of the sampled boundary points. Non-rigid registration parameters for the un-sampled points are interpolated linearly. Such hierarchical approach enables the method to register images efficiently. Moreover, registration of visibly distinct bodily tissue regions provides accurate and robust result in region boundaries and inside the regions.