Breast cancer is one of the most common cancer risks to women in the world. Amongst multiple breast imaging modalities, mammography has been widely used in breast cancer diagnosis and screening. Quantitative analyses including breast boundary segmentation and calcification localization are essential steps in a Computer Aided Diagnosis system based on mammography analysis. Due to uneven signal spatial distributions of pectoral muscle and glandular tissue, plus various artifacts in imaging, it is still challenging to automatically analyze mammogram images with high precision. In this paper, a fully automated pipeline of mammogram image processing is proposed, which estimates skin-air boundary using gradient weight map, detects pectoral-breast boundary by unsupervised pixel-wise labeling with no pre-labeled areas needed, and finally detects calcifications inside the breast region with a novel texture filter. Experimental results on Mammogram Image Analysis Society database show that the proposed method performs breast boundary segmentation and calcification detection with high accuracy of 97.08% and 96.15% respectively, and slightly higher accuracies are achieved on Full-Field Digital Mammography image datasets. Calculation of Jaccard and Dice indexes between segmented breast regions and the ground truths are also included as comprehensive similarity evaluations, which could provide valuable support for mammogram analysis in clinic.