When there is substantial heterogeneity of treatment effectiveness, it is crucial to identify individualized treatment assignment rules for comparative treatment selection. Traditional approaches directly model clinical outcome and define optimal treatment rule according to the interactions between treatment and covariates. This approach relies on the success of separating the main effects from the covariate-treatment interaction effects, which may not be easy. To overcome this shortcoming, a recent approach, called outcome weighted learning, focuses on building an optimal treatment rule by maximizing the expected clinical outcome related with differential treatments. However, there seems to be a lack of approaches to explicitly deal with three or more treatments. In this article, we propose an outcome weighted learning method that extends estimating individualized treatment rules to multi-treatment case by using a vector hinge loss as a target function. Consistency of the resulting estimator is shown in the article. We demonstrate the performance of our approach in simulation studies and in a real data analysis.