In this paper, we propose a new hybrid Local Binary Pattern (LBP) based on Hessian matrix and Attractive Center-Symmetric LBP (ACS-LBP), called Hess-ACS-LBP. The Hessian matrix provides the directional derivative information of different texture regions, while ACS-LBP reveals the local texture features efficiently. To obtain the macro-and micro-structure textural changes, Hessian matrix is calculated in a multiscale schema. Multiscale Hessian matrix presents the intrinsic local geometry of the texture changes. The magnitude information of the Hessian matrix is used in the ACS-LBP method. A cross-scale joint coding strategy is used to construct Hess-ACS-LBP descriptor. Finally, histogram concatenation is carried out. Extensive experiments on eight texture databases of CUReT, USPTex, KTH-TIPS2b, MondialMarmi, OuTeX TC_00013, XU HR, ALOT and STex validate the efficiency of the proposed method. The proposed Hess-ACS-LBP method achieves about 20% improvement over the original LBP method and 1%-11% improvement over the other state-of-the-art hand-crafted LBP methods in terms of classification accuracy. Besides, the experimental results show that the proposed method achieves up to 32% better results than the state-of-the-art deep learning based methods. Especially, the performance of the proposed method on ALOT and STex datasets containing many classes is remarkable. INDEX TERMS Hessian matrix, feature extraction, local binary patterns, texture classification.