This paper presents a method about the modeling of the medium - small diesel engine for the speed control and designs the robust speed control system by the $mu$-synthesis, which has good performance, in spite of the existence of model uncertainities and the external disturbance. We confirmed the validity of the proposed modeling method and the designed control system by $mu$-synthesis through the experimental responses.