Human visual systems are sensitive to noise on the flat intensity area. But it becomes less sensitive on the edge area. Recently, many types of spatially adaptive image restoration methods have been proposed, which employ the above mentioned huan visual characteristics. The present paper presents an adaptive image restoration method, which increases sharpness of the edge region, and smooths noise on the flat intensity area. For edge detection, the proposed method uses the visibility function based on the local variance on each pixel. And it adaptively changes the regularization parameter. More specifically, the image to be restored is divided into a number of steps from the flat area to the edge regio, and then restored by using the finite impulse response constrained least squares filter.