We consider the Abelian-Higgs model in 2+1 dimensions with instanton-monopole defects. This model is closely related to the phases of quantum anti-ferromagnets. In the presence of Z2 preserving monopole operators, there are two confining ground states in the monopole phase, corresponding to the Valence Bond Solid (VBS) phase of quantum magnets. We show that the domain-wall carries a 't Hooft anomaly in this case. The anomaly can be saturated by, e.g., charge-conjugation breaking on the wall or by the domain wall theory becoming gapless (a gapless model that saturates the anomaly is SU (2)1 WZW). Either way the fundamental scalar particles (i.e. spinons) which are confined in the bulk are deconfined on the domain-wall. This Z2 phase can be realized either with spin-1/2 on a rectangular lattice, or spin-1 on a square lattice. In both cases the domain wall contains spin-1/2 particles (which are absent in the bulk). We discuss the possible relation to recent lattice simulations of domain walls in VBS. We further generalize the discussion to Abrikosov-Nielsen-Olsen (ANO) vortices in a dual superconductor of the Abelian-Higgs model in 3+1 dimensions, and to the easy-plane limit of anti-ferromagnets. In the latter case the wall can undergo a variant of the BKT transition (consistent with the anomalies) while the bulk is still gapped. The same is true for the easy-axis limit of anti-ferromagnets. We also touch upon some analogies to Yang-Mills theory.