Handwritten character recognition (HCR) is a mainstream mobile device input method that has attracted significant research interest. Although previous studies have delivered reasonable recognition accuracy, it remains difficult to directly embed the advanced HCR service into mobile device software and obtain excellent but fast results. Cloud computing is a relatively new online computational resource provider which can satisfy the elastic resource requirements of the advanced HCR service with high-recognition accuracy. However, owing to the delay sensitivity of the character recognition service, the performance loss in the traditional cloud virtualization technology (e.g., kernel-based virtual machine (KVM)) may impair the performance. In addition, the improper computational resource scheduling in cloud computing impairs not only the performance but also the resource utilization. Thus, the HCR online service is required to guarantee the performance and improve the resource utilization of the HCR service in cloud computing. To address these problems, in this paper, we propose an HCR container as a service (HCRCaaS) in cloud computing. We address several key contributions: (1) designing an HCR engine on the basis of deep convolution neutral networks as a demo for an advanced HCR engine with better recognition accuracy, (2) providing an isolated lightweight runtime environment for high performance and easy expansion, and (3) designing a greedy resource scheduling algorithm based on the performance evaluation to optimize the resource utilization under a quality of service (QoS) guaranteeing. Experimental results show that our system not only reduces the performance loss compared with traditional cloud computing under the advanced HCR algorithm but also improves the resource utilization appropriately under the QoS guaranteeing. This study also provides a valuable reference for other related studies.