Sedimentary stratigraphic sequences are crucial archives of Earth's geological history, providing significant insights into paleoenvironments, climate changes, tectonic activities, and hydrocarbon reservoirs. However, the complexity of subsurface conditions and incomplete knowledge often introduce substantial uncertainty into stratigraphic interpretations. This paper proposes a comprehensive framework for quantifying, communicating, and analyzing stratigraphic uncertainty by incorporating principles from information theory and stochastic processes. Our methodology integrates Markov chains, Poisson processes, and Markov pure-jump processes to mathematically represent the stochastic nature of stratigraphic units, boundaries, and sequences. We also formulate entropy models aligned with these stochastic processes, establishing a robust foundation for addressing uncertainty. Through detailed case studies across diverse sedimentary environments—such as marine sandstones, braided river deltas, and meandering river systems—our findings reveal several key insights: (1) Stratigraphic states within a sequence can be accurately predicted using the Markov chain model, with entropy and entropy rate serving as effective metrics for gauging sequence predictability; (2) The asymptotic equipartition property theorem indicates that the number of stratigraphic sequences increases exponentially with entropy and sequence length, underscoring the stochastic complexity inherent in stratigraphic sequences; (3) Entropy and entropy rate values allow us to quantitatively distinguish between various sedimentary environments. Additionally, the stationary probability of the Markov pure-jump process aids in quantitatively assessing differences among stratigraphic sequences within similar sedimentary contexts; (4) Quantifying the uncertainty associated with stratigraphic states and their thicknesses provides valuable geological insights, aiding geologists in making informed decisions. We also present a sensitivity analysis of our approach and outline directions for future research. The insights gained from this study underscore the potential of our methodology in enhancing the understanding of stratigraphic sequence uncertainty, facilitating more informed decision-making in related disciplines. This research paves the way for a more quantitative approach to stratigraphy.