We describe Grid-LMM, a general algorithm for fitting linear mixed effect models across a wide range of applications in quantitative genetics, including genome-wide association mapping and heritability estimation. Grid-LMM provides approximate (yet highly accurate) frequentist test statistics or Bayesian posterior summaries in a fraction of the time compared to existing general-purpose methods. Most importantly, Grid-LMM is suitable for genome-wide analyses that account for multiple sources of heterogeneity, such as additive and non-additive genetic variance, spatial heterogeneity, and genotype-environment interactions. We demonstrate that Grid-LMM can increase power in common settings using simulation studies and two real data applications. Grid-LMM is available in an R-package.