Non-terrestrial networks, composed of ground, air, and satellite communications, are considered one of the key components for the Beyond 5G/6G, and optical satellite communication is a fundamental technology to enable high-capacity communications. It is affected by interruptions of optical communications due to clouds on the communication link. A satellite can mitigate the interruption by switching its destination ground station to the other communication available station, though it brings additional delays in establishing optical links. In this study, we propose a ground station selection method using reinforcement learning algorithms to realize a fast and stable satellite-terrestrial optical communication system. We introduce two multi-armed bandit algorithms, Q-learning and Deep Q-learning, for the proposed method. We evaluate them using actual data of the optical satellite communication availability. Our simulation results show that the proposed method with deep Q-learning has the best average throughput. The proposed scheme efficiently follows changes in the state of communication links, and it becomes even better than fixed to ideal best link.