In this study, a noncontact laser ultrasonic system is proposed for rail defect detection. An Nd-Yag pulse laser is used for generation of ultrasonic waves, and the corresponding ultrasonic responses are measured by a laser Doppler vibrometer. For the detection of rail surface damages, the shape of the excitation laser beam is transformed into a line. On the other hand, a point source laser beam is used for the inspection of defects inside a rail head. Then, the interactions of propagating ultrasonic waves with defects are examined using actual rail specimens. Amplitude attenuation was mainly observed for a surface crack, and reflections were most noticeable from an internal damage. Finally, opportunities and challenges associated with real-time rail inspection from a high-speed train are discussed.