In order to investigate the nonlinear dynamic characteristics of the star gear transmission system under wide-frequency excitation, the time-varying mesh stiffness is introduced, and the nonlinear dynamic analysis model of high-speed star gear transmission considering the gyroscopic effect is established by combining the mesh error, tooth surface friction, gear backlash, etc. The nonlinear dynamic response of the star gear transmission system is obtained by solving the problem using the Runge-Kutta method, and the nonlinear dynamic response of the star gear transmission system is further obtained by the bifurcation diagram of the mesh damping ratio of the system under wide-frequency excitation. The nonlinear dynamic response of the star gear transmission system is obtained by using the Runge-Kutta method, and furthermore, the bifurcation diagram of the meshing damping ratio of the system under the excitation is obtained. The results show that the high rotational speed will make the system's period interval decrease, the chaos interval lengthen, and the vibration amplitude become larger.