Hot-mix asphalts exposed to hot weather and high traffic volumes can display rutting distress. A material that can be used to increase the stiffness of asphalt binders is gilsonite. On the other hand, from an environmental point of view, the virgin natural aggregates of asphalt mixtures can be replaced with recycled concrete aggregates. For these reasons, this study modified the asphalt binder with gilsonite by wet-process to improve rutting resistance, and replaced (by mass and volume) part of the coarse fraction of the aggregate with recycled concrete aggregate in two hot-mix asphalts with different gradations. Unlike other studies, a larger experimental phase was used here. Marshall, indirect tensile strength, resilient modulus, permanent deformation, fatigue resistance, and Cantabro tests were performed. An ANOVA test was carried out. If the replacement of the virgin aggregate by recycled concrete aggregates was made by volume, both materials (gilsonite and recycled concrete aggregate) could be used in hot-mix asphalts for thick-asphalt layers in high temperature climates and any level of traffic. The use of both materials in hot-mix asphalts is not recommended for thin-asphalt layers in low temperatures climates. It is not advisable to replace the aggregates by mass.