Nowadays, the viscous damper using high viscosity oil was much to be used for engine shafting system to reduce the excessive additional stress by torsional vibration. In general, it was assumed that the viscous damper could be modelled having only damping coefficient, that is to say, whose stiffness be ignored. But it is found that there exists a jump phenomenon, as a kind of non-linear vibration, in the actual engine shafting system with a damper of high viscosity. Therefore the damper ring and the casing are modelled as two mass elastic system with a complex viscosity. Also, to analyze a non-linear phenomenon, it is assumed that the viscous damper has a linear stiffness coefficient in proportion to the angular amplitude and a non-linear stiffness coefficient in proportion to cube of the angular amplitude. For the analysis, Quasi-Newton method with BFGS(Broyden-Fletcher-Goldfarb-Shanno) formula is used. Both calculated and measured values are provided in this paper which confirm the possibility of applying non-linear theory to engine shafting system with viscous damper.