Many image processing applications such as noise suppression and contrast enhancement are designed to be embedded in multimedia devices such as digital cameras. In order to test the accuracy and robustness of these algorithms they should be evaluated in the context of the problem domain. For nearly three decades many promising algorithms have been tested using a series of images which do not realistically represent the breadth of applications for evaluating image processing algorithms. This paper describes some of the shortcomings of using these worn-out images in algorithm testing, and also the use of simulated defects.