We develop a thermodynamic model describing the binding of RNA binding proteins (RBP) to oligomers in vitro. We apply expectation-maximization to infer the specificity of RBPs, represented as position-specific weight matrices (PWMs), by maximizing the likelihood of RNA Bind’n Seq data from the ENCODE project. We demonstrate that the model can reproduce known specificities for well-studied proteins and that in some cases we predict novel, longer binding motifs. However, the model does not recover all the motifs that are in principle known, indicating that the data is not well explained by a single underlying biophysical model. Our code is publicly available.