When temperatures drop to significantly low levels, road pavements are subjected to thermally-induced stresses, resulting in the appearance of thermal cracking, among other distresses. In these situations, polymers can be used as asphalt binder modifiers to improve certain asphalt binder properties, such as elastic recovery, cohesion, and ductility. Polymers also minimize some of the problems of asphalt mixtures, such as thermal and fatigue cracking and permanent deformation. This work’s objective was to study the behavior of asphalt mixtures at low temperatures, mainly when using modified binders. Thus, three binders were selected and tested: a standard 50/70 penetration grade bitumen and two polymer-modified binders (PMB), obtained by adding, respectively, 2.5% and 5.0% of styrene–butadiene–styrene (SBS) in the 50/70 pen grade bitumen. Then, the PMBs were incorporated into stone mastic asphalt mixtures (namely SMA 11), which were subjected to low-temperature mechanical tests based on the most recent European Standards. The asphalt binders and mixtures evaluated in this work were tested for thermal cracking resistance, creep, elastic recovery, cohesive strength, and ductility strength. Overall, it is concluded that the studied asphalt mixtures with PMB, with just 2.5% SBS, performed adequately at low temperatures down to −20 °C.