Zoom motion estimation of video sequence is very complicated for implementation. In this paper, we propose a method to implement the zoom motion estimation using together the depth camera and color camera. Depth camera obtains the distance information between current block and reference block, then zoom ratio between both blocks is calculated from this distance information. As the reference block is appropriately zoomed by the zoom ratio, the motion estimated difference signal can be reduced. Therefore, the proposed method is possible to increase the accuracy of motion estimation with keeping zoom motion estimation complexity not greater. Simulation was to measure the motion estimation accuracy of the proposed method, we can see the motion estimation error was decreased significantly compared to conventional block matching method.