Referring to the formulation of physical stochastic optimal control of structures and the scheme of optimal polynomial control, a nonlinear stochastic optimal control strategy is developed for a class of structural systems with hysteretic behaviors in the present paper. This control strategy provides an amenable approach to the classical stochastic optimal control strategies, bypasses the dilemma involved in It$hat{o}$-type stochastic differential equations and is applicable to the dynamical systems driven by practical non-stationary and non-white random excitations, such as earthquake ground motions, strong winds and sea waves. The newly developed generalized optimal control policy is integrated in the nonlinear stochastic optimal control scheme so as to logically distribute the controllers and design their parameters associated with control gains. For illustrative purposes, the stochastic optimal controls of two base-excited multi-degree-of-freedom structural systems with hysteretic behavior in Clough bilinear model and Bouc-Wen differential model, respectively, are investigated. Numerical results reveal that a linear control with the 1st-order controller suffices even for the hysteretic structural systems when a control criterion in exceedance probability performance function for designing the weighting matrices is employed. This is practically meaningful due to the nonlinear controllers which may be associated with dynamical instabilities being saved. It is also noted that using the generalized optimal control policy, the maximum control effectiveness with the few number of control devices can be achieved, allowing for a desirable structural performance. It is remarked, meanwhile, that the response process and energy-dissipation behavior of the hysteretic structures are controlled to a certain extent.