The appropriate operation of a tunnel ventilation system provides drivers passing through the tunnel with comfortable and safe driving conditions. Tunnel ventilation involves maintaining CO pollutant concentration and VI (visibility index) under an adequate level with operating highly energy-consuming facilities such as jet-fans. Therefore, it is significant to have an efficient operating algorithm in aspects of a safe driving environment as well as saving energy. In this research, a reinforcement learning (RL) method based on the actor-critic architecture and nonparametric policy gradients is applied as the control algorithm. The two objectives listed above, maintaining an adequate level of pollutants and minimizing power consumption, are included into a reward formulation that is a performance index to be maximized in the RL methodology. In this paper, a nonparametric approach is adopted as a promising route to perform a rigorous gradient search in a function space of policies to improve the efficacy of the actor module. Extensive simulation studies performed with real data collected from an existing tunnel system confirm that with the suggested algorithm, the control purposes were well accomplished and improved when compared to a previously developed RL-based control algorithm.