Molecular communication (MC) is a new communication engineering paradigm where molecules are employed as information carriers. MC systems are expected to enable new revolutionary applications such as sensing of target substances in biotechnology, smart drug delivery in medicine, and monitoring of oil pipelines or chemical reactors in industrial settings. As for any other kind of communication, simple yet sufficiently accurate channel models are needed for the design, analysis, and efficient operation of MC systems. In this paper, we provide a tutorial review on mathematical channel modeling for diffusive MC systems. The considered end-to-end MC channel models incorporate the effects of the release mechanism, the MC environment, and the reception mechanism on the observed information molecules. Thereby, the various existing models for the different components of an MC system are presented under a common framework and the underlying biological, chemical, and physical phenomena are discussed. Deterministic models characterizing the expected number of molecules observed at the receiver and statistical models characterizing the actual number of observed molecules are developed. In addition, we provide channel models for timevarying MC systems with moving transmitters and receivers, which are relevant for advanced applications such as smart drug delivery with mobile nanomachines. For complex scenarios, where simple MC channel models cannot be obtained from first principles, we investigate simulation-driven and experimentallydriven channel models. Finally, we provide a detailed discussion of potential challenges, open research problems, and future directions in channel modeling for diffusive MC systems.