This paper presents reconstruction of disparity in images. To achieve this, the algorithm was made up of two different procedures - one is extraction of boundaries for man-made structures and the other is matching of the structures. In the extraction of boundaries for man-made structures, we assume that man-made structures are composed of lines and the lines make up closed polygon. The convertional algorithms of the edges extraction may not perceive man-made structures and have problems that matching algorithms were too complex. This paper proposed sub-pixel boundaries extraction algorithm that fused split-and-merge and image improvement algorithms to overcome complexity. In matching procedure, feature-based algorithm that minimize the proposed cost function are used and the cost fuction considers movement of mid-points for left and right images to match structures. Because we could not obtain disparity of inner parts for the man-made structures, interpolation method was used. The experiment showed good results.