In order to consider high-order effects on the actual limit state function, a new response surface method is proposed for structural reliability analysis by the use of high-order approximation concept in this study. Hermite polynomials are used to determine the highest orders of input random variables, and the sampling points for the determination of highest orders are located on Gaussian points of Gauss-Hermite integration. The cross terms between two random variables, only in case that their corresponding percent contributions to the total variation of limit state function are significant, will be added to the response surface function to improve the approximation accuracy. As a result, significant reduction in computational cost is achieved with this strategy. Due to the addition of cross terms, the additional sampling points, laid on two-dimensional Gaussian points off axis on the plane of two significant variables, are required to determine the coefficients of the approximated limit state function. All available sampling points are employed to construct the final response surface function. Then, Monte Carlo Simulation is carried out on the final approximation response surface function to estimate the failure probability. Due to the use of high order polynomial, the proposed method is more accurate than the traditional second-order or linear response surface method. It also provides much more efficient solutions than the available high-order response surface method with less loss in accuracy. The efficiency and the accuracy of the proposed method compared with those of various response surface methods available are illustrated by five numerical examples.