The occurrence of railway accidents is not always the result of error/violation on the operator’s end; rather, there are many organizational factors involved. This research investigated 97 railway accident reports from the West Zone of Bangladesh Railway between 2015 and 2019 to categorize factors within the institution that contribute to accidents. The accidents were divided into four categories (collision, derailment, locomotive failure, and signal passed at danger [SPAD]) and were applied to the Human Factors Analysis and Classification System (HFACS) framework. The HFACS framework systematically identifies direct and hidden factors that lead to an accident. A total of 225 human and organizational factors were found involved in 97 accidents, of which 106 (47.11%) were organizational-level factors and 119 (52.89%) were front-line/operation-level factors. To understand the association between factors, a Bayesian network was created following the naive Bayes algorithm, keeping “accident type” as the class variable. Analysis revealed that “Collisions” occur because of organizational factors such as the structure of command, planned inappropriate operations, and operation-level factors such as perceptual errors, and personal unreadiness. “Derailment” was influenced by organizational reasons, such as equipment issues, company safety culture, and inadequate supervision, whereas operation-level factors were crew and resource management and skill-based error. “Locomotive failure” was found to be associated with only organizational factors, such as equipment issues and inadequate supervision, and SPAD mostly occurred because of exceptional violations at the operational level. The identified issues, if addressed properly, can help ensure a safer railway industry.