Mobile devices and their novel applications are handling increasing amounts of new type of content such as video, images, messages and music. Managing large amounts of data introduces a usability challenge that cannot be solved without developing new ways of managing the content. The use of context metadata is presented here as one of the ways to aid content management. This paper investigates the usability issues that are introduced by the usage of context metadata such as time, date, location and proximity. The research was based on literature, expert groups, focus groups and interviews. The usability issues that were identified were related to meaningfulness, to usefulness and to privacy. A vast amount of issues need to be solved to implement a safe, easy-to-use and preferably automated way of assisting users to manage their mobile content with context metadata.