In nowadays, the infrared optics is frequently employed to various fields such as military, aerospace, industry and medical. To develop the infrared optics, special glasses which can transmit infrared wave are required. Ge(Germanium), Si(silicon), and fluoride glasses are typically used for material of the infrared optics. Compared with Ge and Si glasses, fluoride glasses have high transmittance in infrared wavelength range. Additionally, UV(ultraviolet) and visible light can be transmitted through fluoride glasses. There characteristics of fluoride glasses makes it possible to evaluate optical performance with generally used visible testing equipment. In this paper, we used design of experiment to find ultra precision machining characteristic of Ge and fluoride glasses and optimized machining process to obtain required form accuracy of PV(Peak to Valley) $0.2;{mu}m$.