The Gasper and Rahman multivariate (−q)-Racah polynomials appear as connection coefficients between bases diagonalizing different abelian subalgebras of the recently defined higher rank q-Bannai-Ito algebra A q n . Lifting the action of the algebra to the connection coefficients, we find a realization of A q n by means of difference operators. This provides an algebraic interpretation for the bispectrality of the multivariate (−q)-Racah polynomials, as was established in Iliev, Trans. Amer. Math. Soc., 363(3) (2011), 1577-1598.Furthermore, we extend the Bannai-Ito orthogonal polynomials to multiple variables and use these to express the connection coefficients for the q = 1 higher rank Bannai-Ito algebra An, thereby proving a conjecture from De Bie et al., Adv. Math. 303 (2016), 390-414. We derive the orthogonality relation of these multivariate Bannai-Ito polynomials and provide a discrete realization for An.