The optimal output tracking control (OOTC) problem for a class of discrete-time systems with state and input delays is addressed. An augmented system is constructed such that the OOTC problem can be transformed into a two-point boundary value (TPBV) problem with both advance and delay terms from the necessary optimality conditions. The successive approximating method recently developed is extended to obtain an approximate solution of the TPBV problem, which is then used to obtain a feedforward and feedback tracking controller. An observer is designed for the uncertain reference input such that the feedforward controller is physically realizable. Simulations show the results are effective even with long time-delays.