Abstract-As the diversity in end-user devices and networks grows, it becomes important to be able to efficiently and adaptively serve media content to different types of users. A key question surrounding adaptive media is how to do RateDistortion optimized scheduling. Typically, distortion is measured with a single distortion measure, such as the Mean-Squared Error compared to the original high resolution image or video sequence. Due to the growing diversity of users with varying capabilities such as different display sizes and resolutions, we introduce Multiple Distortion Measures to account for a diverse range of users and target devices. Multiple Distortion Measures (MDM) gives a clear framework with which to evaluate the performance of media systems which serve a variety of users. Scalable coders, such as JPEG2000 and H.264/MPEG-4 SVC, allow for adaptation to be performed with relatively low computational cost. We show that accounting for MDM can significantly improve system performance; furthermore, by combining this with scalable coding, this can be done efficiently. Given these Multiple Distortion Measures, we propose an algorithm to generate embedded schedules, which enables low-complexity, adaptive streaming of scalable media packets to minimize distortion across multiple users. We show that using MDM achieves up to 4dB gains for spatial scalability applied to images and 12dB gains for temporal scalability applied to video.Index Terms-Multiple distortion measures, scalable streaming, embedded packet schedules, rate-distortion optimization, JPEG2000, H.264/MPEG-4 SVC