Wireless sensor networks (WSNs) have taken a giant leap in scale, expanding their applicability to a large variety of technological domains and applications, ranging from the Internet of things (IoT) for smart cities and smart homes to wearable technology healthcare applications, underwater, agricultural and environmental monitoring and many more. This expansion is rapidly growing every passing day in terms of the variety, heterogeneity and the number of devices which such applications support. Data collection is commonly the core application in WSN and IoT networks, which are typically composed of a large variety of devices, some constrained by their resources (e.g., processing, storage, energy) and some by highly diverse demands. Many challenges span all the conceptual communication layers, from the Physical to the Applicational. Many novel solutions devised in the past do not scale well with the exponential growth in the population of the devices and need to be adapted, revised, or new innovative solutions are required to comply with this massive growth. Furthermore, recent technological advances present new opportunities which can be leveraged in this context. This paper provides a cross-layer perspective and review of data gathering in WSN and IoT networks. We provide some background and essential milestones that have laid the foundation of many subsequent solutions suggested over the years. We mainly concentrate on recent state-of-the-art research, which facilitates the scalable, energy-efficient, cost-effective, and human-friendly functionality of WSNs and the novel applications in the years to come.