We derive, using functional methods and the bias expansion, the conditional likelihood for observing a specific tracer field given an underlying matter field. This likelihood is necessary for Bayesian-inference methods. If we neglect all stochastic terms apart from the ones appearing in the auto two-point function of tracers, we recover the result of Schmidt et al., 2018 [1]. We then rigorously derive the corrections to this result, such as those coming from a non-Gaussian stochasticity (which include the stochastic corrections to the tracer bispectrum) and higher-derivative terms. We discuss how these corrections can affect current applications of Bayesian inference. We comment on possible extensions to our result, with a particular eye towards primordial non-Gaussianity. This work puts on solid theoretical grounds the EFT-based approach to Bayesian forward modeling. arXiv:1909.04022v1 [astro-ph.CO] 9 Sep 2019 1 Even if we use an N-body simulation to forward-model the matter field, the evolution is never deterministic (due to the fact that, by construction, N-body simulations still integrate out all sub-grid modes), so the assumption of having a Dirac delta functional is not technically correct. We will see below that this is naturally taken into account by the approach used in this paper. of this paper is to compute P[δ g |δ] using the effective field theory of biased tracers.Let us outline our general strategy to compute P[δ g |δ]. First, we can use the properties of conditional probabilities to express this as the ratio of the joint probability P[δ g , δ] and the probability P[δ] for the matter field itself. How do we compute these two quantities? We know how to compute correlation functions of δ g and δ using the effective field theory of large-scale structure (EFT of LSS)/bias expansion. More generally, we know how to compute the full generating functionals Z[J] and Z[J g , J], the first for the correlation functions of δ alone, and the second for those of δ g and δ together. The generating functional for matter correlation functions in the EFT of LSS was discussed for the first time (to the authors' knowledge) in [13]: here we extend it for the first time to include also biased tracers.Once we have the two generating functionals Z[J] and Z[J g , J], an expression for P[δ] and P[δ g , δ] can be easily obtained via the inverse functional Fourier transform. This allows us to cast the problem of computing these probability density functionals (henceforth "likelihoods", for simplicity) using methods of functional integration commonly employed in quantum field theory. As we will see below, this method successfully recovers the result of [1], where the authors computed P[δ g |δ] using the assumption that the noise field ε g (that enters in the linear bias expansion of δ g as δ g = b 1 δ + ε g ) is a Gaussian-distributed variable.On top of this, the approach developed here gives a simple and rigorous way of computing the corrections to this result which come, for example, from the wrong assumption of having Gauss...