In this paper, we propose an efficient algorithm for adaptation of tracking windwo, which improves tracking performance of a correlation-based video tracker by rejecting background effect originated from a time-varying target. Th eproposed adaptation algorithm ajdusts the size of a tracking window by using the ratio of spatial gradient power in target region to that in backgorund region, which is especially adequate for a correlation-based tracker. Experimental results for synthetic and real image sequences show that the proposed method adapts a tracking window well to a time-varying target and so greatly suppresses background effect, which makes improvement of trakcing performance.