In this paper, the depth controller of an underwater vehicle based on an $H_infty$ servo control is designed for the depth keeping of the underwater vehicle under wave disturbances. The depth controller is designed in the form of the $H_infty$ servo controller, which has robust tracking property, and an $H_infty$ servo problem is considered for the $H_infty$ servo controller design. In order to solve the $H_infty$ servo problem for the underwater vehicle, this problem is modified as an $H_infty$ control problem for the generalized plant that includes a reference input mode, and a suboptimal solution that satisfies a given performance criteria is calculated with the LMI (Linear Matrix Inequality) approach. The $H_infty$ servo controller is designed to have robust stability about the perturbation of the parameters of the underwater vehicle and the robust tracking property of the underwater vehicle depth under wave force and moment disturbances. The performance, robustness about the uncertainties, and depth tracking property, of the designed depth controller is evaluated by computer simulation, and finally these simulation results show the usefulness and applicability of the proposed $H_infty$ depth control system.