This paper presents a novel Markov switching state space control model for dynamically switching resource configuration scheme to achieve power conservation for multimedia server cluster systems. This model exploits the hierarchical dynamic structure of network system and its construction is flexible and scalable. Using this analytical model, the problem of power conservation is posed as a constrained stochastic optimization problem with the goal of minimizing the average power consumption subject to the constraint on the average blocking ratio. Applying Lagrange approach and online estimation of the performance gradient, a policy iteration algorithm is proposed to search the optimal policy online. This algorithm does not depend on any prior knowledge of system parameters, and converges to the optimal solution. Simulation results demonstrate the convergence of the proposed algorithm and effectiveness to different access workloads.