An agent-based model (ABM) is a computational model in which the local interactions of autonomous agents with each other and with their environment give rise to global properties within a given domain. As the detail and complexity of these models has grown, so too has the computational expense of running several simulations to perform sensitivity analysis and evaluate long-term model behavior. Here, we generalize a framework for mathematically formalizing ABMs to explicitly incorporate features commonly found in biological systems: appearance of agents (birth), removal of agents (death), and locally dependent state changes. We then use our broader framework to extend an approach for estimating long-term behavior without simulations, specifically changes in population densities over time. The approach is probabilistic and relies on treating the discrete, incremental update of an ABM via "time steps" as a Markov process to generate expected values for agents at each time step. As case studies, we apply our extensions to both a simple ABM based on the Game of Life and a published ABM of rib development in vertebrates.