As a classical and significant problem, handwritten character recognition has been widely used in our daily lives. With recent deep learning methods, previous studies have achieved a great improvement for this problem in the past few years. However, the handwritten character recognition for Urdu, which is one of the largest languages of the world, is less studied in the existing literature. In this paper, we fill in this gap and evaluate different deep learning models on the problem of Urdu handwritten characters recognition based on a newly released dataset. Combined with data augmentation and transfer learning techniques, we achieve the state-of-the-art results by recognizing digits and characters with an accuracy of 98.94% and 99.08%, respectively, which greatly improves the baselines of 97% and 86.5%.