The presence of expandable clay minerals within the reservoir and source rocks causes many issues over the life of the well. These include wellbore instability and increased mud losses during drilling operations, complications and decrease in production during and after hydraulic fracturing, etc. Existing standard laboratory test methods, e.g., capillary suction time test, do not adequately predict swelling potential. The present work provides the technique to determine the properties and quantify the concentration of expandable clay minerals, thus predicting the swelling potential of clay-rich rocks. A collection of ash layer and bentonite mine samples has been analyzed for the bulk and clay fraction mineral composition and swelling in completion fluids. Comparison of X-ray diffractograms recorded from air-dried and ethylene glycolated clay fractions of ash layer samples confirmed the absence of discrete smectite but revealed the presence of interstratified illite/smectite and illite. X-ray diffractograms of all the ethylene glycolated samples have characteristic complex reflections with two maximas: one between 6.8–7.8°2θ (11.3–13.1 Å), another between 9.3–9.6°2θ (9.3–9.5 Å). Both correspond to the 001 reflection of interstratified illite/smectite mineral, and not a pure endmember illite or smectite. Two properties of this mineral vary among the samples and influence their expandability: (1) illite:smectite ratio in the range 3:2–9:1, and (2) the nature of the interstratification, i.e., the degree of ordering. The shifts of the peak maximas in the regions 6.8–9.6°2θ and 42.0–48.0°2θ are used to calculate the proportion of the component layers. Proposed technique of optimized XRD analysis reveals that the samples containing less illite/smectite mineral, but the higher concentration of smectite layers in this mixed-layer mineral are more expandable. Optimized XRD analysis correctly predicts the swelling potential of a given clay-rich zone (e.g., ash layer) as confirmed by confined immersion testing. Provided technique of XRD analysis is fast, requires small quantities of sample, and could possibly be integrated to field logs in the future. Obtained knowledge can be applied by engineers and researchers working with shale formations around the world.