Many techniques are available for destructive and nondestructive measurement of mechanical hardness. The ultrasonic method could be widely applicable as a nondestructive technique. Many studies have examined how changes in the mechanical hardness affect the longitudinal velocity of ultrasonic waves. This approach aims to estimate the overall velocity variations in specimens. However, proper nondestructive examination techniques are needed as effective tools for analyzing the effects of heat treatment on the surface of the specimens. Therefore, in this study, the effect on the hardness of piston rods was nondestructively measured using surface ultrasonic waves. The hardness after heat treatment was investigated at various depths in the specimens, and the velocity of the surface ultrasonic waves was measured with respect to the hardness of the piston rods. In addition, finite element method simulations were performed to confirm the behavior of the waves.