In Mendelian randomization (MR) analysis, variants that exert horizontal pleiotropy are typically treated as a nuisance. However, they could be valuable in identifying novel pathways to the traits under investigation. Here, we developed the MR-TRYX framework, following the advice of William Bateson to "TReasure Your eXceptions". We begin by detecting outliers in a single exposure-outcome MR analysis, hypothesising they are due to horizontal pleiotropy. We search across thousands of complete GWAS summary datasets in the MR-Base database to systematically identify other ("candidate") traits that associate with the outliers. We developed a multi-trait pleiotropy model of the heterogeneity in the exposure-outcome analysis due to pathways through candidate traits. Through detailed investigation of several causal relationships, many pleiotropic pathways were uncovered with already established causal effects, validating the approach, but also novel putative causal pathways. Adjustment for pleiotropic pathways reduced the heterogeneity across the analyses.