In the field of image recognition, circle detection is one of the most widely used techniques. Conventional algorithms are mainly based on Hough transform, which is the most straightforward algorithm for detecting circles and for providing enough robust algorithm. However, it suffers from large memory requirements and high computational loads, and sometimes tends to detect incorrect circles. This paper proposes an optimal circle detection and approximation method which is applicable for inspecting fiber optic connector endface. The proposed method finds initial center coordinates and radius based on the initial edge lines. Then, by introducing the simplified K-means algorithm, the proposed method investigates a substitute-circle by minimizing the area of non-overlapped regions. Through extensive simulations, it is shown that the proposed method can improve the error rate by as much as 67% and also can reduce the computing time by as much as 80%, compared to the Hough transform provided by the OpenCV library.