The possibility to understand and to quantitatively model the physics of the interactions between pedestrians walking in crowds has compelling relevant applications, e.g. related to the efficient design and the safety of civil infrastructures. In this work we study pedestrian-pedestrian interactions from observational experimental data in diluted pedestrian crowds. While in motion, pedestrians continuously adapt their walking paths trying to preserve mutual comfort distances and to avoid collisions. In mathematical models this behavior is typically modeled via "social" interaction forces.Leveraging on a high-quality, high-statistics dataset -composed of few millions of real-life trajectories acquired from state of the art observational experiments (about 6 months of high-resolution pedestrian tracks acquired in a train station) -we develop a quantitative model capable of addressing interactions in the case of binary collision avoidance. We model interactions in terms of both long-range (sight based) and short-range (hard-contact avoidance) forces, which we superimpose to our Langevin model for non-interacting pedestrian motion [Corbetta et al. Phys. Rev. E 95, 032316, 2017] (here further tested and extended). The new model that we propose here features a Langevin dynamics with "fast" random velocity fluctuations that are superimposed to the "slow" dynamics of a hidden model variable: the "intended" walking path. In case of interactions, social forces may act both on the intended path and on the actual walked path. The model is capable of reproducing quantitatively relevant statistics of the collision avoidance motion, such as the statistics of the side displacement and of the passing speed. Rare occurrences of actual bumping events are also recovered.Furthermore, comparing with large datasets of real-life tracks involves an additional computational challenge so far neglected: identifying automatically, within a database containing very heterogeneous conditions, only the relevant events corresponding to binary avoidance interactions. In order to tackle this challenge, we propose a novel and general approach based on a graph representation of pedestrian trajectories, which allows us to effectively operate complexity reduction for efficient data classification and selection.