In the steel manufacturing area, researches for defect inspection receive a big attention for quality control. This paper proposes an algorithm to detect a scratch defect on steel billets. This algorithm takes ROIs (Regions of Interest), and extracts 11 features which represent properties of defect on a ROI. SVM (Support Vector Machine) is used to classify defect and normal ROIs. The algorithm classifies a frame image of a Billet as a defect image if there is one or more defect ROIs. In the experiments, the proposed algorithm had reliable classifying accuracy.