This article is intended to present a multibody dynamic model of the drilling system, consisting of drillstring and drilling fluid. The drillstring is a complex rigid-flexible coupling system, including rigid bodies, Euler-Bernoulli beam elements, constraints and dynamic loads, and its dynamic model is established using the absolute nodal coordinate formulation. The drilling fluid, composed of internal, annulus, and under-bit fluids, is modeled as one-dimensional compressible fluid; the relative flow of the drilling fluid is modeled using the Arbitrary Lagrangian-Eulerian description; the force of the drillstring acting on the drilling fluid is introduced through the drilling fluid transport motion; meanwhile, the reaction force acting on the drillstring is taken as an external load. The contact between the drillstring and drilling fluid is simulated based on Hertz contact theory, and the rock penetration model is built based on the rock-breaking velocity equation. Based on this model, the coupled vibration of the drillstring and the effects of the drilling fluid flow rate and density on the drilling process are investigated through several examples.