Internet of Things (IoT) offers a seamless platform to connect people and objects to one another for enriching and making our lives easier. This vision carries us from compute-based centralized schemes to a more distributed environment offering a vast amount of applications such as smart wearables, smart home, smart mobility, and smart cities. In this paper we discuss applicability of IoT in healthcare and medicine by presenting a holistic architecture of IoT eHealth ecosystem. Healthcare is becoming increasingly difficult to manage due to insufficient and less effective healthcare services to meet the increasing demands of rising aging population with chronic diseases. We propose that this requires a transition from the clinic-centric treatment to patient-centric healthcare where each agent such as hospital, patient, and services are seamlessly connected to each other. This patient-centric IoT eHealth ecosystem needs a multi-layer architecture: 1) device, 2) fog computing and 3) cloud to empower handling of complex data in terms of its variety, speed, and latency. This fog-driven IoT architecture is followed by various case examples of services and applications that are implemented on those layers. Those examples range from mobile health, assisted living, e-medicine, implants, early warning systems, to population monitoring in smart cities. We then finally address the challenges of IoT eHealth such as data management, scalability, regulations, interoperability, device-network-human interfaces, security, and privacy.