The degenerate affine and affine BMW algebras arise naturally in the context of Schur-Weyl duality for orthogonal and symplectic Lie algebras and quantum groups, respectively. Cyclotomic BMW algebras, affine Hecke algebras, cyclotomic Hecke algebras, and their degenerate versions are quotients. In this paper the theory is unified by treating the orthogonal and symplectic cases simultaneously; we make an exact parallel between the degenerate affine and affine cases via a new algebra which takes the role of the affine braid group for the degenerate setting. A main result of this paper is an identification of the centers of the affine and degenerate affine BMW algebras in terms of rings of symmetric functions which satisfy a "cancellation property" or "wheel condition" (in the degenerate case, a reformulation of a result of Nazarov). Miraculously, these same rings also arise in Schubert calculus, as the cohomology and K-theory of isotropic Grassmanians and symplectic loop Grassmanians. We also establish new intertwiner-like identities which, when projected to the center, produce the recursions for central elements given previously by Nazarov for degenerate affine BMW algebras, and by Beliakova-Blanchet for affine BMW algebras.