Higher life expectancy is increasing the number of age-related cognitive impairment cases. It is also relevant, as some authors claim, that physical exercise may be considered as an adjunctive therapy to improve cognition and memory after strokes. Thus, the integration of physical and cognitive therapies could offer potential benefits. In addition, in general these therapies are usually considered boring, so it is important to include some features that improve the motivation of patients. As a result, computer-assisted cognitive rehabilitation systems and serious games for health are more and more present. In order to achieve a continuous, efficient and sustainable rehabilitation of patients, they will have to be carried out as part of the rehabilitation in their own home. However, current home systems lack the therapist’s presence, and this leads to two major challenges for such systems. First, they need sensors and actuators that compensate for the absence of the therapist’s eyes and hands. Second, the system needs to capture and apply the therapist’s expertise. With this aim, and based on our previous proposals, we propose an ambient intelligence environment for cognitive rehabilitation at home, combining physical and cognitive activities, by implementing a Fuzzy Inference System (FIS) that gathers, as far as possible, the knowledge of a rehabilitation expert. Moreover, smart sensors and actuators will attempt to make up for the absence of the therapist. Furthermore, the proposed system will feature a remote monitoring tool, so that the therapist can supervise the patients’ exercises. Finally, an evaluation will be presented where experts in the rehabilitation field showed their satisfaction with the proposed system.