Abstract. Classification is one of the core problems in computer-aided cancer diagnosis (CAD) via medical image interpretation. High detection sensitivity with reasonably low false positive (FP) rate is essential for any CAD system to be accepted as a valuable or even indispensable tool in radiologists' workflow. In this paper, we propose a novel classification framework based on sparse representation. It first builds an overcomplete dictionary of atoms for each class via K-SVD learning, then classification is formulated as sparse coding which can be solved efficiently. This representation naturally generalizes for both binary and multiwise classification problems, and can be used as a standalone classifier or integrated with an existing decision system. Our method is extensively validated in CAD systems for both colorectal polyp and lung nodule detection, using hospital scale, multi-site clinical datasets. The results show that we achieve superior classification performance than existing stateof-the-arts, using support vector machine (SVM) and its variants [1, 2], boosting [3], logistic regression [4], relevance vector machine (RVM) [5,6], or k-nearest neighbor (KNN) [7].