The presented research is focused on the development and testing of the magnesium potassium phosphate cement-based materials (MKPC-based). Firstly, the fresh state properties of the pastes consisting of dead burned magnesia powder, potassium dihydrogen phosphate, setting retarder borax applied in the range of 0–10 wt.%, and batch water were investigated. The aim of testing was to characterize the hydration process in dependence on the borax content. The properties of raw MgO powder were described by chemical composition and particle size distribution. The properties tested in fresh state included shear stress (viscosity), Young’s modulus of elasticity, and temperature; their time dependence was observed. The measurements started immediately after the mixing process. At the age of 14 days, basic structural and mechanical properties of the hardened pastes were obtained. The mixture with 5 wt.% of borax proved to be the most advantageous in terms of setting time, sample integrity, and mechanical strength; therefore, it was chosen as the binder for the following part of the study—MKPC-based mortar development. In the next step, the MKPC paste containing 5 wt.% of borax was supplemented by silica sand aggregate, and the resulting material was marked as a reference. Subsequently, three other mixtures were derived by replacing 100% of quartz sand by lightweight aggregate; namely by expanded glass aggregate, waste rubber from tires, and combination of both in ratio 1:1. The aggregates were characterized by chemical composition (except for the rubber granulate), and loose and compacted powder density. For the resulting hardened composites, basic structural, hygric, strength, and thermal parameters were investigated. The use of lightweight aggregates brought in a considerable decrease in heat transport parameters and low water permeability while maintaining sufficient strength. The favorable obtained material properties are underscored by the fact that magnesia-phosphate is considered to be a low-carbon binder. The combination of magnesia-phosphate binder and recycled aggregate provides a satisfying, environmentally friendly, and thermally efficient alternative to traditional Portland cement-based materials.