A flexible load can vary its power consumption to perform grid support services. This flexibility is naturally limited by the Quality of Service (QoS) requirements at the load. A widely examined class of flexible loads is Thermostatically Controlled Loads (TCLs), which include air conditioners, water heaters, and refrigerators. A TCL is designed to maintain a temperature within a preset band, and the actuation to achieve this is on/off. Temperature, cycling rate, and the energy bill are three main QoS metrics: exceeding the temperature limits, frequent cycling between on and off, and a high energy bill must be avoided.How the temperature constraint affects the capacity of an ensemble of TCLs to provide grid support is a well studied problem. However, how the cycling constraint effects the capacity is often neglected. In this work we present a characterization of the capacity of a collection of TCLs that takes into account not only temperature, but also cycling and energy constraints. Our characterization of capacity is consistent with its most practical utility: a grid authority can use this characterization to plan a reference signal that the TCLs can track without violating any of their QoS constraints. Additionally, the proposed characterization is independent of the algorithm used to coordinate the TCLs (to provide grid support) and leads to a convex and feasible optimization problem for the grid authority's reference planning.