This paper investigates the accuracy of high-order extended finite element methods (XFEMs) for the solution of discontinuous problems with both straight and curved weak discontinuities in two dimensions. The modified XFEM, a specific form of the stable generalised finite element method, is found to offer advantages in cost and complexity over other approaches, but suffers from suboptimal rates of convergence due to spurious higher-order contributions to the approximation space. An improved modified XFEM is presented, with basis functions "corrected" by projecting out higher-order contributions that cannot be represented by the standard finite element basis. The resulting corrections are independent of the equations being solved and need be pre-computed only once for geometric elements of a given order. An accurate numerical integration scheme that correctly integrates functions with curved discontinuities is also presented. Optimal rates of convergence are then recovered for Poisson problems with both straight and quadratically curved discontinuities for approximations up to order p ≤ 4. These are the first truly optimal convergence results achieved using the XFEM for a curved weak discontinuity and are also the first optimally convergent results achieved using the modified XFEM for any problem with approximations of order p > 1. Almost optimal rates of convergence are recovered for an elastic problem with a circular weak discontinuity for approximations up to order p ≤ 4.