In TFT-LCD panel images, defects are typically detected by using a large difference in the brightness compared to the background. In this paper, we propose a sequential defect detection algorithm according to defect possibility caused by difference of brightness. By using this method, pixels with high defect probabilities are preferentially detected and defects with a large brightness difference are accurately detected. Also, limited defects with a small brightness difference is detected more reliably, eventually minimizing the degree of over-detection. We have experimentally confirmed that our proposed method showed an excellent detection result for detecting limited defects as well as defects with a large brightness difference.