It is known that statistical model selection as well as identification of dynamical equations from available data are both very challenging tasks. Physical systems behave according to their underlying dynamical equations which, in turn, can be identified from experimental data. Explaining data requires selecting mathematical models that best capture the data regularities. The existence of fundamental links among physical systems, dynamical equations, experimental data and statistical modeling motivate us to present in this article our theoretical modeling scheme which combines information geometry and inductive inference methods to provide a probabilistic description of complex systems in the presence of limited information. Special focus is devoted to describe the role of our entropic information geometric complexity measure. In particular, we provide several illustrative examples wherein our modeling scheme is used to infer macroscopic predictions when only partial knowledge of the microscopic nature of a given system is available. Finally, limitations, possible improvements, and future investigations are discussed.