The Acousto-Optic Tunable Filter (AOTF) is a high-speed full-field monochromator which generates two spectrally filtered light beams with ordinary and extraordinary polarization state. Thus, AOTF is widely used to build full-field spectral imaging system or spectral domain interferometer. However, AOTF has a big problem that the angle of diffracted light changes according to the scanning of wavelength, which makes image shift on CCD plane In this paper, we propose an analytic design of prism system to compensate the image shift. The detailed analysis of light paths in a prism and basic experimental results are presented to verify our proposed compensation method. The experimental results agree with simulation results based on suggested prism model and image shift is minimized at optimal condition. Also, it can be extended to compensate the image shift for ordinary and extraordinary polarized light simultaneously.