The interaction among pedestrians and human drivers is a complicated process, in which road users have to communicate their intentions, as well as understand and anticipate the actions of users in their vicinity. However, road users still ought to have a proper interpretation of each others’ behaviors, when approaching and crossing the road. Pedestrians, as one of the interactive agents, demonstrate different behaviors at road crossings, which do not follow a consistent pattern and may vary from one situation to another. The presented inconsistency and unpredictability of pedestrian road crossing behaviors may thus become a challenge for the design of emerging technologies in the near future, such as automated driving system (ADS). As a result, the current paper aims at understanding the effectual communication techniques, as well as the factors influencing pedestrian negotiation and decision-making process. After reviewing the state-of-the-art and identifying research gaps with regards to vehicle–pedestrian crossing encounters, a holistic approach for road crossing interaction modeling is presented and discussed. It is envisioned that the presented holistic approach will result in enhanced safety, sustainability, and effectiveness of pedestrian road crossings.