The diffusion of particles has wide repercussions, ranging from particle-based soft-matter systems to solid-state systems with particular electronic properties. Recently, in the field of magnetism, the diffusion of magnetic skyrmions, topologically stabilized quasiparticles, has been demonstrated. Here, we show that, by applying a magnetic in-plane field, and therefore, breaking the symmetry of the system, skyrmion diffusion becomes anisotropic, with faster diffusion parallel to the field axis and slower diffusion perpendicular to it. We furthermore show that the absolute value of the applied magnetic in-plane field controls the absolute values of the diffusion coefficients, so that one can thereby tune both the orientation of the diffusion and its strength. Based on the stochastic Thiele equation, we can explain the observed anisotropic diffusion as a result of the elliptical deformation of the skyrmions by the application of the in-plane field.