A CFD analysis method is developed and applied for investigating the gas flow and the byproduct particle trajectory in Roots type vacuum pump. The internal fluid flow and thermal fields between the rotors and the housing of vacuum pump are analyzed by using the dynamic mesh, the numerical methods for unsteady 2-D Navier-Stokes equation and the standard k-$varepsilon$ turbulence model of the Fluent code. Coupled with the flow simulation results, the particle trajectory of the byproduct flowing into the pump with gas stream is analyzed by using discrete phase modeling technique. The CFD analysis results show the pressure, the velocity and the temperature distributions in pump change abruptly due to the rotation of rotors, and back flows are produced due to the strong reverse pressure gradients at rotor/rotor and rotor/housing clearances. The predicted byproduct particle trajectory results also show the particles impinge on the clearance surfaces between the housing and the rotor of pump and then may form the deposit layer causing the failure of pump.