Chemical composition analysis, chromatography and spectroscopy are dominate quality evaluation methods for agarwood, which are cumbersome and time-consuming. To facilitate its quality evaluation, a global-to-local optimization method is proposed to automatically inspect the appearance of the burned agarwood stick. First, a dissimilarity coefficient is defined by the attributes of the connected domains to coarsely localize the carbon line region. Then, the threshold for the coarsely localized carbon line region is adaptively determined based on grayscale characteristics of image patches partitioned from the coarsely localized carbon line region. Next, the threshold is used to extract the contour of the carbon line region and to establish the fine localization model for locally and precisely localizing the carbon line region. Finally, an ash shrinkage compensation coefficient is defined to calculate the ash shrinkage rate (ASR). The ASR combined with carbon line height is utilized to characterize the appearance of burned agarwood. Experimental results indicate that the proposed inspection method can well detect the carbon line regions and ashes of burned agarwood sticks, with a mean ASR error of 0.74%, which is superior to some existing inspection methods.