By their very nature Spin Waves (SWs) enable the realization of energy efficient circuits, as they propagate and interfere within waveguides without consuming noticeable energy. However, SW computing can be even more energy efficient by taking advantage of the approximate computing paradigm as many applications, e.g., multimedia and social media, are error-tolerant. In this paper, we propose an ultra-low energy Approximate Full Adder (AFA) and an Approximate 2-bit inputs Multiplier (AMUL). AFA consists of one Majority gate whereas AMUL is built by means of 3 AND gates. We validate the correct functionality of our proposal by means of micromagnetic simulations and evaluate AFA's figures of merit against state-of-the-art accurate SW, 7nm CMOS, Spin Hall Effect (SHE), Domain Wall Motion (DWM), accurate and approximate 45nm CMOS, Magnetic Tunnel Junction (MTJ), and Spin-CMOS FA implementations. Our results indicate that AFA consumes 38% and 6% less energy than state-of-the-art accurate SW and 7nm CMOS FA implementations, respectively. Moreover, it saves 56% and 20% energy when compared with accurate and approximate 45nm CMOS counterparts, respectively. Furthermore, it provides 2 orders of magnitude energy reduction when compared with accurate SHE, accurate and approximate DWM, MTJ, and Spin-CMOS, counterparts. In addition, it achieves the same error rate as approximate 45nm CMOS and Spin-CMOS FAs whereas it exhibits 50% less error rate than the approximate DWM FA. Last but not least, it outperforms its contenders in terms of area by saving at least 29% chip real-estate. AMUL is evaluated and compared with state-of-the-art SW and 16nm CMOS accurate and approximate designs. The evaluation results indicate that AMUL energy consumption is at least 2.8x and 2.6x smaller than the one of state-of-the-art SW and 16nm CMOS accurate and approximate designs, respectively. AMUL has an error rate of 25%, whereas the approximate CMOS multiplier has an error rate of 38%, and requires at least 64% less chip real-estate than the CMOS counterpart.