In this paper the continuous utility representation problem will be discussed in arbitrary concrete categories. In particular, generalizations of the utility representation theorems of Eilenberg, Debreu and Estévez and Hervés will be presented that also hold if the codomain of a utility function is an arbitrary totally ordered set and not just the real line. In addition, we shall prove and apply a general result on the characterization of structures that have the property that every continuous total preorder has a continuous utility representation. Finally, generalizations of the utility representation theorems of Debreu and Eilenberg will be discussed that are valid if we consider arbitrary binary relations and allow a utility function to have values in an arbitrary totally ordered set.