The identification of drilling joint dynamics which consists of drilling and structural dynamics and the on-line time series detection of malfunction process is substantial not only for the investigation of the static and dynamic characteristics but also for the analytic realization of diagnostic and control systems in drilling. Therefore, We have discussed on the comparative assessment of two recursive time series modeling algorithms that can represent the drilling operation and detect the abnormal geometric behaviors in precision roundshape machining such as turning, drilling and boring in precision diemaking. For this purpose, simulation and experimental work were performed to show the malfunctional behaviors for drilling operation. For this purpose, a new two recursive approach (Recursive Extended Instrument Variable Method : REIVM, Recursive Least Square Method : RLSM) may be adopted for the on-line system identification and monitoring of a malfunction behavior of drilling process, such as chipping, wear, chatter and hole lobe waviness.