We generalize the concept of basin of attraction of a stable state in order to facilitate the analysis of dynamical systems with noise and to assess stability properties of metastable states and long transients. To this end we examine the notions of mean sojourn times and absorption probabilities for Markov chains and study their relation to the basins of attraction. Our approach is directly applicable to all systems that can be approximated as Markov chains, including stochastic and deterministic differential equations. We also provide a sampling based generalization of basin stability that works without resorting to the Markov approximation by sampling trajectories directly.We discuss two far reaching generalizations of the basin of attraction of an attractor. These apply to general sets in the phase space of nondeterministic systems. The first is based on absorption probabilities, the second on expected mean sojourn times with respect to a finite time horizon. By casting the problem in the transfer operator language, we are able to give a simple formula for the first generalization along the lines of committor functions.We show that the two notions of generalized basin coincide in the limit of a vanishing absorption probability and an infinite time horizon respectively. Importantly, for well-behaved deterministic systems this limit recovers the usual notion of basin of attraction. Finally we point out that derived quantities like the volume of the generalized basin are accessible through sampling trajectories at the same computational cost as evaluating basin stability for deterministic systems.