In developing an automated surface inspect algorithm, we have designed a hierarchical classifier using neural network. The defects which exist on the surface of cold mill strip have a scattering or singular distribution. We have considered three major problems, that is preprocessing, feature extraction and defect classification. In preprocessing, Top-hit transform, adaptive thresholding, thinning and noise rejection are used Especially, Top-hit transform using local minimax operation diminishes the effect of bad lighting. In feature extraction, geometric, moment, co-occurrence matrix, and histogram ratio features are calculated. The histogram ratio feature is taken from the gray-level image. For defect classification, we suggest a hierarchical structure of which nodes are multilayer neural network classifiers. The proposed algorithm reduced error rate by comparing to one-stage structure.