Significant environmental issues are raised by the phosphogypsum (PG) waste that is being produced. In Lithuania, about 1,500,000 tons of PG waste is generated yearly, and about 300 Mt is generated yearly worldwide. A by-product of burning wood biomass in thermal power plants is biomass fly ash (BFA). By 2035, compared to 2008 levels, industrial biomass incineration for combined heat and power and, as a consequence, BFA, is expected to triple. This study revealed the possibility of using these difficult-to-utilise waste products, such as BFA and PG, in efficient alkali-activated materials (AAM). As the alkaline activator solution (AAS), less alkaline Na2CO3 solution and Na2SiO3 solution were used. The study compared the physical–mechanical properties of BFA-PG specimens mixed with water and the AAS. After 28 days of curing, the compressive strength of the BFA-PG-based, water-mixed samples increased from 3.02 to 6.38 MPa when the PG content was increased from 0 to 30 wt.%. In contrast, the compressive strength of the BFA-PG-based samples with AAS increased from 8.03 to 16.67 MPa when the PG content was increased from 0 to 30 wt.%. According to XRD analysis, gypsum crystallisation increased when the PG content in the BFA-PG-based samples with water increased. The presence of AAS in the BFA-PG-based samples significantly reduced gypsum crystallisation, but increased the crystallisation of the new phases kottenheimite and sodium aluminium silicate hydrate, which, due to the sodium ions’ participation in the reactions, created denser reaction products and improved the mechanical properties. The outcome of this investigation aids in producing sustainable AAM and applying high volume of hardly usable waste materials, such as BFA and PG.