The possibility of matter coupling to two metrics at once is considered. This appears natural in the most general ghost-free, bimetric theory of gravity, where it unlocks an additional symmetry with respect to the exchange of the metrics. This double coupling, however, raises the problem of identifying the observables of the theory. It is shown that if the two metrics couple minimally to matter, then there is no physical metric to which all matter would universally couple, and that moreover such an effective metric generically does not exist even for an individual matter species. By studying point particle dynamics, a resolution is suggested in the context of Finsler geometry.PACS 04.20. Cv, 04.80.Cc, 98.80.Jk, 95.30.Sf, The seemingly innocuous modification of Einstein's general theory of relativity (GR) by adding a tiny mass to the graviton may entail radically new perspectives on the nature of spacetime and its interplay with matter. In the context of GR, the graviton is identified with the fluctuations of the metric describing the curvature of spacetime. To give these fluctuations a mass, we need to introduce an external metric and are thus led