Eye movement is a key aspect of human perception and interaction, with the extraocular muscles (EOMs) providing the primary driving forces for three-dimensional rotations of the eyeball. However, existing biomechanical models of the coupled EOM-eyeball system struggle to accurately account for the complex wrapping and contact interactions between the muscles and the eyeball, and the equations of motion for the eyeball often neglect its large-amplitude rotational characteristics. In this work, a geometric mechanics framework is developed to construct a dynamic model of the coupled EOM-eyeball system, aiming to reveal the biomechanical mechanisms by which active EOM forces induce three-dimensional ocular rotations. First, the geodesic method is employed to determine the three-dimensional wrapping paths between the EOMs and the eyeball, and a model to describe the variation of EOM moment arms during large-amplitude horizontal eye movements is established. Second, by treating the eyeball as a rigid body with a fixed center of rotation, the Euler-Lagrange equations of motion are derived on the SO(3) Lie group. On this basis, the rotational matrices are interpolated on SO(3), and and ocular torques are obtained via inverse dynamics based on the Euler-Lagrange formulation. Furthermore, a static optimization algorithm is introduced to estimate the optimal activation patterns of the extraocular muscle group. The proposed approach is validated using the open-source simulation software OrbitTM 1.8 and OpenSim. The results show that the EOM paths obtained by the geodesic method are consistent with those simulated in OrbitTM 1.8, and that the calculated moment arms and optimized muscle activation patterns agree well with the OpenSim results. The geometric mechanics and control framework developed herein provides a theoretical model and quantitative basis for understanding the biomechanical mechanisms by which the extraocular muscles control eye movements, as well as for clarifying the pathological mechanisms and therapeutic strategies of extraocular muscle-related disorders such as strabismus.