The search and rescue (SAR) scenario is complex and uncertain where a robot needs to understand the scenario to make smart decisions. Aiming at the knowledge representation (KR) in the field of SAR, this paper builds an ontology model that enables a robot to understand how to make smart decisions. The ontology is divided into three parts, namely entity ontology, environment ontology, and task ontology. Web Ontology Language (OWL) is adopted to represent these three types of ontology. Through ontology and Semantic Web Rule Language (SWRL) rules, the robot infers the tasks to be performed according to the environment state and at the same time obtains the semantic information of the victims. Then, the paper proposes an ontology-based algorithm for task planning to get a sequence of atomic actions so as to complete the high-level inferred task. In addition, an indoor experiment was designed and built for the SAR scenario using a real robot platform—TurtleBot3. The correctness and usability of the ontology and the proposed methods are verified by experiments.