In this paper, an algorithm that finds man-made structures in a praylevel aerial images is proposed to perform stereo matching. An extracted contour of buildings must have a high accuracy in order to get a good feature-based stereo matching result. Therefore this study focuses on the use of edge following in the original image rather than use of ordinary edge filters. The Algorithm is composed of two main categories; one is to find candidate regions in the whole image and the other is to extract exact contours of each building which each candidate region.. The region growing method using the centroid linkage method of variance value is used to find candidate regions of building and the contour line tracing algorithm based on an adge following method is used to extract exact contours. The result shows that the almost contours of building composed of line segments are extracted.