The recent advent of high precision antihydrogen (H) spectroscopy opens the way to stringent experimental tests of the fundamental principles underlying particle physics and general relativity (GR), such as Lorentz and CPT invariance and the Einstein Equivalence Principle (EEP), on pure antimatter systems. In this paper, the nature and implications of these tests is investigated, with special reference to the ALPHA antihydrogen programme at CERN. This is underpinned by a theoretical review of the role of antiparticles, causality and fundamental symmetries in relativistic quantum field theory (QFT) and the theory of time measurement in GR. Low-energy effective theories which break Lorentz and CPT symmetry, or the Strong Equivalence Principle (SEP), are then introduced, together with a review of several 'fifth force' scenarios involving new long-range forces which would effectively violate the universality of free-fall, or Weak Equivalence Principle (WEPff). The possible role of CPT violation in determining the matter-antimatter asymmetry of the Universe is discussed. Explicit calculations are given for the dependence on possible Lorentz and CPT violating couplings of the transition frequencies amongst the 1S, 2S and 2P hyperfine states measured in the magnetic field of the ALPHA trap and the resulting bounds are compared with existing limits. An analysis of the implications for the EEP of current free-fall and spectroscopic measurements with antihydrogen is presented and existing and potential bounds on WEPff and the universality of clocks (WEPc) are derived, together with constraints on fifth forces. Future prospects for high-precision antihydrogen spectroscopy, free-fall and gravitational redshift experiments, and antiatom matter-wave interferometry are described and experimental possibilities involving other antimatter species are briefly outlined.