Sand mining, among the many activities that have significant effects on the bed changes of rivers, has increased in many parts of the world in recent decades. Numerical modeling plays a vital role in simulation in the long term; however, computational time remains a challenge. In this paper, we propose a sand mining component integrated into the bedload continuity equation and combine it with high-performance computing using graphics processing units to boost the speed of the simulation. The developed numerical model is applied to the Mekong river segment, flowing through Tan Chau Town, An Giang Province, Vietnam. The 20 years from 1999 to 2019 is examined in this study, both with and without sand mining activities. The results show that the numerical model can simulate the bed change for the period from 1999 to 2019. By adding the sand mining component (2002–2006), the bed change in the river is modeled closely after the actual development. The Tan An sand mine in the area (2002–2006) caused the channel to deviate slightly from that of An Giang and created a slight erosion channel in 2006 (−23 m). From 2006 to 2014, although Tan An mine stopped operating, the riverbed recovered quite slowly with a small accretion rate (0.25 m/year). However, the Tan An sand mine eroded again from 2014–2019 due to a lack of sand. In 2014, in the Vinh Hoa communes, An Giang Province, the Vinh Hoa sand mine began to operate. The results of simulating with sand mining incidents proved that sand mining caused the erosion channel to move towards the sand mines, and the erosion speed was faster when there was no sand mining. Combined with high-performance computing, harnessing the power of accelerators such as graphics processing units (GPUs) can help run numerical simulations up to 23x times faster.