For image contrast enhancement operation, it is a keypoint to obtain more natural enhanced results and keep more details without distortion. In this paper, a novel image Hessian-matrix weighted histogram for image contrast enhancement is proposed, which can improve the contrast of smooth regions and simultaneously restrain the contrast of texture regions. In the proposed method, the multi-scale fractional-order Hessian-matrix is firstly utilized to detect and quantify the texture information of the input image, which explores the regions that should be contrasted or should be restrained. Then, the strong texture regions are suppressed by a designed suppress function. Finally, the information on unsuppressed regions and suppressed texture regions will be count by a histogram, which is termed as Hessian-matrix weighted Histogram (HessHist) in this paper. According to HessHist, the corresponding cumulative distribution function will realize the contrast enhancement operation on the input image. For real-time application, the integral images are introduced for fast computation of the HessHist. Experimental results show that the proposed HessHist-based image enhancement algorithm preserves more details of input image without distortion, and is competitive with state-of-the-art image enhancement algorithms in both subjective visual perception and objective evaluation metrics.