Coarse-grained potentials are ubiquitous in mesoscale simulations. While various methods to compute effective interactions for spherically symmetric particles exist, anisotropic interactions are seldom used, due to their complexity. Here we describe a general formulation, based on a spatial decomposition of the density fields around the particles, akin to atomic orbitals. We show that anisotropic potentials can be efficiently computed in numerical simulations using Fourier-based methods. We validate the field formulation and characterize its computational efficiency with a system of colloids that have Gaussian surface charge distributions. We also investigate the phase behavior of charged Janus colloids immersed in screened media, with screening lengths comparable to the colloid size. The system shows rich behaviors, exhibiting vapor, liquid, gel, and crystalline morphologies, depending on temperature and screening length. The crystalline phase only appears for symmetric Janus particles. For very short screening lengths, the system undergoes a direct transition from a vapor to a crystal on cooling; while, for longer screening lengths, a vapor-liquid-crystal transition is observed. The proposed formulation can be extended to model force fields that are time or orientation dependent, such as those in systems of polymer-grafted particles and magnetic colloids.