Aims. GAIA delivers the positions and velocities of stars at an unprecedented precision. Therefore, for star clusters, there exists much higher confidence in whether a specific star is a member of a particular cluster or not. However, membership determination is still especially challenging for young star clusters. At ages 2-10 Myr, the gas is expelled, ending the star formation process and leading to their expansion, while at the same time, many former members become unbound. As a first step, we aim to assess the accuracy of the methods commonly used to distinguish between bound and unbound cluster members; after identifying the most suitable technique for this task, we wish to understand which of the two populations is more suited to provide insights into the initial configuration and the dynamical history of a cluster starting from its currently observed properties. Methods. Here, we perform N-body simulations of the dynamics of such young star clusters. We investigate how cluster dynamics and observational limitations affect the recovered information about the cluster from a theoretical perspective. Results. We find that the much-used method of distance and velocity cutoffs for membership determination often leads to false negatives and positives alike. Often observational studies focus on the stars remaining bound. However, bound stars quickly lose the memory of the pre-gas expulsion phase due to their ongoing interaction with their fellow cluster members. Our study shows that it is the unbound stars that hold the key to charting a cluster's dynamic history. Backtracking unbound stars can provide the original cluster size and determine the time of gas expulsion -two parameters that are currently still poorly constrained. This information is lost in the bound population. In addition, former members are often better indicators for disc lifetimes or initial binary fractions. We apply the backtracking analysis, with varying success, to the clusters: Upper Scorpius and NGC 6530. For highly substructured clusters such as Upper Scorpius, backtracking to the individual subcluster centres will provide better results in future.