The recently-developed off-line learning control approaches for the rejection of periodic disturbances utilize the specific property that the learning system tends to oscillate in steady state. Unfortunately, the prior works have not clarified how closely the learning system should approach the steady state to achieve the rejection of periodic disturbances to satisfactory level. In this paper, we address this issue extensively for the class of linear systems. We also attempt to remove the effect of other aperiodic disturbances on the rejection of the periodic disturbances effectively. In fact, the proposed learning control algorithm can provide very fast convergence performance in the presence of aperiodic disturbance. The effectiveness and practicality of our work is demonstrated through mathematical performance analysis as well as various simulation results.