Web service composition (WSC) is the task of generating new composite web services that exhibit functionalities not supported by any single web service. In its simplest form this is achieved by linking existing web services in sequence. More complex forms link services in parallel or use alternative paths. WSC can be considered a planning task, with the web services being the planning operators and the initial state and the goals being provided by the user. Particularly, since web services operate in a stochastic environment, their output is not predictable, and the problem is formulated as a non-deterministic planning one. This article presents a critical, comprehensive and up-to-date review of the literature concerning alternative non-deterministic planning methods, including probabilistic planning, determinization methods, planning in the belief state space and translationbased methods. Furthermore, the article reviews existing implementations of WSC systems, employing a variety of planning approaches, and discusses the degree in which the current achievements from the non-deterministic planning field have been adopted successfully. To the best of our knowledge, this is the first review of its kind, one that provides a thorough introduction to the vast area of automated web service composition.