Abstract. With the rapid development of web service technology, next generations of web service applications need to be able to predict problems, such as potential degradation scenarios, future erroneous behaviors and deviations from expected behaviors, and move towards resolving those problems not just reactively, but even proactively, i.e., before the problems occur. Service oriented applications are thus driven by the requirements that bring the concepts of decentralization, dynamism, adaptation, and automation to an extreme. In this paper, an approach is proposed that depends mainly on the concept of proactive adaptation by the use of reinforcement learning to achieve an autonomous dynamic behavior of web service composition considering potential degradation and emergence in QoS values. Experimental results show the effectiveness of the proposed approach in dynamic web service composition environments.