A novel interaction method for mobile phones using their built-in cameras is presented. By estimating the path connecting the center points of frames captured by the camera phone, objects of interest can be easily extracted and recognized. To estimate the movement of the mobile phone, corners and corresponding Speeded-Up Robust Features descriptors are used to calculate the spatial transformation parameters between the previous and current frames. These parameters are then used to recalculate the locations of the center points in the previous frame into the current frame. The experiment results obtained from real image sequences show that the proposed system is efficient, flexible, and able to provide accurate and stable results.