We present a new hybrid stereo matching technique in terms of the co-operation of area-based stereo and feature-based stereo. The core of our technique is that feature matching is carried out by the reference of the disparity evaluated by area-based stereo. Since the reference of the disparity can significantly reduce the number of feature matching combinations, feature matching error can be drastically minimized. One requirement of the disparity to be referenced is that it should be reliable to be used in feature matching. To measure the reliability of the disparity, in this paper, we employ the self-consistency of the disunity Our suggested technique is applied to the detection of 3D line segments by 2D line matching using our hybrid stereo matching, which can be efficiently utilized in the generation of the rooftop model from urban imagery. We carry out the experiments on our hybrid stereo matching scheme. We generate synthetic images by photo-realistic simulation on Avenches data set of Ascona aerial images. Experimental results indicate that the extracted 3D line segments have an average error of 0.5m and verify our proposed scheme. In order to apply our method to the generation of 3D model in urban imagery, we carry out Preliminary experiments for rooftop generation. Since occlusions are occurred around the outlines of buildings, we experimentally suggested multi-image hybrid stereo system, based on the fusion of 3D line segments. In terms of the simple domain-specific 3D grouping scheme, we notice that an accurate 3D rooftop model can be generated. In this context, we expect that an extended 3D grouping scheme using our hybrid technique can be efficiently applied to the construction of 3D models with more general types of building rooftops.