Multi-hop Knowledge Base Question Answering (KBQA) aims to find the answer entities in a Knowledge Base (KB) that are multiple hops away from the topic entities mentioned in a question. The key challenge of multi-hop KBQA is the significant semantic gap between Natural Language (NL) and the relation paths in KB. Recently, the Case-Based Reasoning (CBR) approach has been successfully applied to KBQA, which first retrieves other similar questions from the training set and then referencing their solutions (e.g., reasoning path) to derive a solution for given question. However, existing CBR-based methods are hard to provide correct solutions while the gold reasoning paths are unavailable. Moreover, they are limited to the assumption that the reasoning path required to answer a question is repeats for similar questions, thus suffer from the generalization issue. To address above problems, we propose a novel framework, which consider both the semantic matching between NL question and KB paths, as well as the matching between KB paths of similar question and given question, to alleviate the inconsistency between NL and KB. Specifically, it uses above two matching mechanisms to generate more accurate solutions for training questions and then reuses it to rank the candidate reasoning paths of given question. To cope with the generalization issue mentioned above, our method adopts different ranking algorithm according to different NL similarity. Experiments over two benchmarks demonstrate the effectiveness of our approach, which achieves state of the art on MetaQA and new state of the art results on PathQuestion-Large.