Applications based on synergistic integration of optical imagery and LiDAR data are receiving a growing interest from the remote sensing community. However, a misaligned integration between these datasets may fail to fully profit from the potential of both sensors. In this regard, an optimum fusion of optical imagery and LiDAR data requires an accurate registration. This is a complex problem since a versatile solution is still missing, especially when considering the context where data are collected at different times, from different platforms, under different acquisition configurations. This paper presents a coarse-to-fine registration method of aerial or satellite optical imagery with airborne LiDAR data acquired in such context. Firstly, a coarse registration involves processes of extraction and matching of building candidates from LiDAR data and optical imagery. Then, a Mutual Information-based fine registration is carried out. It involves a super-resolution approach applied to LiDAR data to generate images with the same resolution as the optical image, and a local approach of transformation model estimation. The proposed method succeeds at overcoming the challenges associated with the aforementioned difficult context. For instance, considering the experimented airborne LiDAR (2011) and orthorectified aerial imagery (2016) datasets, their spatial shift is reduced by almost a half (i.e. 48.15%) after the proposed coarse registration. Moreover, the incompatibility of size and spatial resolution is well addressed by the mentioned super-resolution approach. Finally, a high accuracy of dataset alignment is also achieved, highlighted by a 40-cm error based on a check-point assessment and a 64-cm error based on a checkpair-line assessment. Promising results yielded by this registration enable further research for a complete versatile fusion methodology between airborne LiDAR and optical imagery data in this challenging context.