We study the topologically twisted compactification of the 6d (2, 0) M5-brane theory on an elliptically fibered Kähler three-fold preserving two supercharges. We show that upon reducing on the elliptic fiber, the 4d theory is N = 4 Super-Yang Mills, with varying complexified coupling τ , in the presence of defects. For abelian gauge group this agrees with the so-called duality twisted theory, and we determine a non-abelian generalization to U(N ). When the elliptic fibration is singular, the 4d theory contains 3d walls (along the branch-cuts of τ ) and 2d surface defects, around which the 4d theory undergoes SL(2, Z) duality transformations. Such duality defects carry chiral fields, which from the 6d point of view arise as modes of the two-form B in the tensor multiplet. Each duality defect has a flavor symmetry associated to it, which is encoded in the structure of the singular elliptic fiber above the defect. Generically 2d surface defects will intersect in points in 4d, where there is an enhanced flavor symmetry. The 6d point of view provides a complete characterization of this 4d-3d-2d-0d 'Matroshka'-defect configuration.