As the prefabrication construction method plays an increasingly important role in the construction of cross‐sea bridges, coordinating the schedule between off‐site prefabrication and on‐site assembly becomes crucial and challenging. This paper studies a just‐in‐time single‐machine scheduling problem with due windows and release dates, which originates from the production and supply process of large segments of precast steel box girders in bridge construction. To solve this problem, we equivalently decompose it into the independent and same type of subproblems and formulate a mixed‐integer linear mathematical programming model. Furthermore, we propose a branch‐and‐bound algorithm with the initialization of a novel linear early and tardy dispatching rule with due windows to solve the problem exactly. The dispatching rule, based on the local dominance criteria with due windows, is designed and used to improve the upper bound. Moreover, a lower bound is obtained by further decomposing and relaxing the subproblems. Finally, the parameter of the dispatching rule is calibrated, and the best search strategy of the branch‐and‐bound algorithm is determined. The comprehensive computational experiments carried out show the efficiency and effectiveness of the proposed branch‐and‐bound algorithm.