The aim of this paper is to develop a methodology for measuring the degree of unpredictability in dynamical systems with memory, i.e., systems with responses dependent on a history of past states. The proposed model is generic, and can be employed in a variety of settings, although its applicability here is examined in the particular context of an industrial environment: gas turbine engines. The given approach consists in approximating the probability distribution of the outputs of a system with a deep recurrent neural network; such networks are capable of exploiting the memory in the system for enhanced forecasting capability. Once the probability distribution is retrieved, the entropy or missing information about the underlying process is computed, which is interpreted as the uncertainty with respect to the system's behaviour. Hence the model identifies how far the system dynamics are from its typical response, in order to evaluate the system reliability and to predict system faults and/or normal accidents. The validity of the model is verified with sensor data recorded from commissioning gas turbines, belonging to normal and faulty conditions.