A family of high entropy oxides with the formula Mg2Ta3Ln3O14 (Ln = La, Pr, Nd, Sm, Eu, Gd) has been discovered and synthesized. Single crystals, 5 mm OD × 2.5 cm, for Ln = Nd have been grown using the laser optical floating zone technique. Crystal orientations are confirmed by Laue diffraction, and structure solutions were obtained via single crystal X-ray diffraction. The structure is found to be a partially disordered pyrochlore, space group Fd-3m, fractional chemical formula (Mg0.25Nd0.75)2(Mg0.25Ta0.75)2O7. Magnetization measurements indicate ordinary paramagnetic behavior in all compounds down to T = 2 K, except in the Eu variant which possesses Van Vleck paramagnetism. Specific heat measurements for Ln = Nd shows no phase transitions between T = 300 and 2 K. We demonstrate the ability to prepare magnetically disordered materials by substitution of Mg with Ni, Mn, and Co, demonstrating the flexibility of this family in accommodating defects. The stabilization of these compounds could be due to the entropy gain associated with defects, showcasing a “materials by design” approach by using disorder to stabilize novel magnetic and optical materials. Our work also demonstrates the feasibility of preparing high entropy oxides in single crystalline form.