Nowadays, harvesting, sorting, and packaging fruit and vegetables are still done manually, despite the hard work this represents. The features that experts commonly use to sorting the date palm fruit are size, color, shape, and texture. Recently, it has started to design and develop artificial vision systems that consider the criteria of size, color, shape, and texture to automate these processes. However, the development of these systems is complex due to the lack of labeled datasets that facilitate the creation of models to locate, recognize and classify palm date fruit. This dataset is entitled Medjool, an image dataset of different sizes and maturity levels of Medjool dates. Researchers may use this data to develop a model for automatic location, recognition, classification, and visual counting of the Medjool dates on trays taking into account their visual features such as shape, color, size, and texture. This dataset was collected from the first-round harvest at Palmeras RQ Ranch in Mexicali, Mexico. Images acquisition was performed in natural light. The dataset comprises 2,576 annotated images in two formats, YOLO and PascalVOC format.