Modern cyber-physical systems would often fall victim to unanticipated anomalies. Humans are still required in many operations to troubleshoot and respond to such anomalies, such those in future deep space habitats. To maximize the effectiveness and efficiency of the anomaly response process, the information provided by anomaly response technologies to their human operators must be epistemically accessible or explainable. This paper offers a first step towards developing explainable anomaly response systems. It proposes a logic, Causal Signal Temporal Logic (CaSTL), which can formally describe causeeffect relationships pertaining to fault explanation. Moreover, it develops an algorithm to infer a CaSTL formula that explains why a fault has happened in a system, given the model of the system and an observation about the fault. The effectiveness of the proposed algorithm is demonstrated with a simulated Environmental Control and Life Support System (ECLSS).