Microbiome engineering offers the potential to leverage microbial communities to improve outcomes in human health, agriculture, and climate. To translate this potential into reality, it is crucial to reliably predict community composition and function. But a brute force approach to cataloguing community function is hindered by the combinatorial explosion in the number of ways we can combine microbial species. An alternative is to parametrize microbial community outcomes using simplified, mechanistic models, and then extrapolate these models beyond where we have sampled. But these approaches remain data-hungry, as well as requiring ana priorispecification of what kinds of mechanism are included and which are omitted. Here, we resolve both issues by introducing a new, mechanism-agnostic approach to predicting microbial community compositions using limited data. The critical step is the discovery of a sparse representation of the community landscape. We then leverage this sparsity to predict community compositions, drawing from techniques in compressive sensing. We validate this approach onin silicocommunity data, generated from a theoretical model. By sampling just ∼ 1% of all possible communities, we accurately predict community compositions out of sample. We then demonstrate the real-world application of our approach by applying it to four experimental datasets, and showing that we can recover interpretable, accurate predictions from highly limited data.