We study the effect of a magnetic field in the Kondo regime of a double-quantum-dot system consisting of a strongly correlated dot (the "side dot") coupled to a second, noninteracting dot that also connects two external leads. We show, using the numerical renormalization group, that application of an in-plane magnetic field sets up a subtle interplay between electronic interference, Kondo physics, and Zeeman splitting with nontrivial consequences for spectral and transport properties. The value of the side-dot spectral function at the Fermi level exhibits a nonuniversal field dependence that can be understood using a form of the Friedel sum rule that appropriately accounts for the presence of an energy-and spin-dependent hybridization function. The applied field also accentuates the exchange-mediated interdot coupling, which dominates the ground state at intermediate fields leading to the formation of antiparallel magnetic moments on the dots. By tuning gate voltages and the magnetic field, one can achieve complete spin polarization of the linear conductance between the leads, raising the prospect of applications of the device as a highly tunable spin filter. The system's low-energy properties are qualitatively unchanged by the presence of weak on-site Coulomb repulsion within the second dot.