This paper presents a novel structural damage detection method with a new damage index based on the statistical moments of dynamic responses of a structure under a random excitation. After a brief introduction to statistical moment theory, the principle of the new method is put forward in terms of a single-degree-of-freedom (SDOF) system. The sensitivity of statistical moment to structural damage is discussed for various types of structural responses and different orders of statistical moment. The formulae for statistical moment-based damage detection are derived. The effect of measurement noise on damage detection is ascertained. The new damage index and the proposed statistical moment-based damage detection method are then extended to multi-degree-of-freedom (MDOF) systems with resort to the leastsquares method. As numerical studies, the proposed method is applied to both single and multi-story shear buildings. Numerical results show that the fourth-order statistical moment of story drifts is a more sensitive indicator to structural stiffness reduction than the natural frequencies, the second order moment of story drift, and the fourth-order moments of velocity and acceleration responses of the shear building. The fourth-order statistical moment of story drifts can be used to accurately identify both location and severity of structural stiffness reduction of the shear building. Furthermore, a significant advantage of the proposed damage detection method lies in that it is insensitive to measurement noise.