This letter presents a very simple yet very effective solution for fast reference frame (RF) selection in H.264/AVC. By efficiently making use of the correlation between the best RF indices in various inter modes, the proposed method significantly reduces the number of RFs to be examined at the expense of a very small miss rate. Simulation results show that the proposed method not only improves upon the coding performance of conventional methods but also reduces the encoding time significantly.