Sediment compositions and trace element (Cu, Pb, Zn, Cr, Fe, Mn, As, and Hg) concentrations were analyzed to assess the sediment quality and the factors affecting the trace elements’ concentration in Ha Tinh province’s coastal sediments. Results revealed that the concentration of trace elements in the coastal sediments was generally lower than in other coastal areas in Vietnam. The significantly positive correlation of the fine sediment composition with Cu, Pb, Zn, Cr, and Fe indicated that the silt and clay sediments could absorb these trace elements. The highly positive correlation among Cu, Pb, Zn, Cr, and Fe suggested that the trace elements likely originated from similar sources. The sediment quality assessment showed that trace element concentrations were lower than the limited values in the sediment quality guideline (QCVN 43:2017/BTNMT). According to the Igeo index, the sediments were not polluted by Cu, Pb, Zn, Cr, Fe, and As but were likely polluted by Hg and Mn in some sampling sites. Therefore, it is necessary to invest the future studies to assess the Hg and Mn concentration levels in these sites to protect the marine environment.