Selenium, a trace element of significant importance for human health and the environment, can be introduced into the environment through coal combustion. Accurate determination of selenium in coal and coal-bearing strata is essential for implementing effective management strategies and control measures to minimize potential risks to human health and the environment. This study introduces an improved approach for the determination of 77Se in the medium resolution mode using HR-ICP-MS, effectively separating interference from doubly charged ions and enabling precise determination of selenium in coal-bearing strata. The relative errors of the standard reference samples obtained by HR-ICP-MS are between 0.65% and 6.33%, comparing to that of ICP-CCT-MS (1.58%–17.27%), prove the reliability of this method. Additionally, the X (bar)—S control charts obtained from HR-ICP-MS compared to ICP-CCT-MS demonstrate the superior stability of HR-ICP-MS in continuous determination. Consequently, though ICP-CCT-MS has better instrumental stability reflected through the internal standard recovery (ICP-CCT-MS:104.81%; HR-ICP-MS:80.54%), HR-ICP-MS is recommended as the preferred method for selenium determination in coal-bearing strata because of its high accuracy and good stability.