This In this paper, we propose an efficient algorithm for motion estimation of the image sequences with luminance fluctuation. For such sequences, conventional motion estimation methods based on the difference of pixel values usually produce the erroneous motion information. The proposed algorithm defines the luminance fluctuation as a linear model with gain and offset parameter, and extracts motion information using gradient and phase of the corresponding local region within consecutive frames. Therefor the method is robust to the luminance change of the frames. We test our algorithm for the ground truth sequence with artificially added luminance change and motion, and real sequences corrupted by the flicker. The results shows that the proposed algorithm outperforms the conventional methods.