This paper proposes a novel Bayesian strategy for high-dimensional inverse problems in the context of elastostatics. Apart from parametric uncertainties, model inadequacies and, particularly, constitutive model errors, are also addressed. This is especially important in biomedical settings when the inferred material properties will be used to make decisions/diagnoses. Traditional approaches use an additional regression model (e.g., Gaussian process), added to the model output or within a submodel to account, for an underlying model error. This can violate physical constraints and becomes impractical in high dimensions.In this work we unfold conservation and constitutive laws to estimate model discrepancies. In addition, efficient Bayesian strategies are employed. In elastography, the accurate identification of the unknown mechanical properties of a tissue as well as the associated uncertainty, can greatly assist noninvasive, medical diagnosis.