Over the last 10 years, 40% of approved oral drugs exhibited a significant effect of food on their pharmacokinetics (PK) and currently the only method to characterize the effect of food on drug absorption, which is recognized by the authorities, is to conduct a clinical evaluation. Within the pharmaceutical industry, there is a significant effort to predict the mechanism and clinical relevance of a food effect. Physiologically based pharmacokinetic (PBPK) models combining both drug-specific and physiology-specific data have been used to predict the effect of food on absorption and to reveal the underlying mechanisms. This manuscript provides detailed descriptions of how a middle-out modeling approach, combining bottom-up in vitro-based predictions with limited top-down fitting of key model parameters for clinical data, can be successfully used to predict the magnitude and direction of food effect when it is predicted poorly by a bottom-up approach. For nefazodone, a mechanistic clearance for the gut and liver was added, for furosemide, an absorption window was introduced, and for aprepitant, the biorelevant solubility was refined using multiple solubility measurements. In all cases, these adjustments were supported by literature data and showcased a rational approach to assess the factors limiting absorption and exposure.