Clofazimine (CLZ) is an effective antibiotic used against a wide spectrum of Gram-positive bacteria and leprosy. One of its main drawbacks is its poor solubility in water. Silica based materials are used as drug delivery carriers that can increase the solubility of different hydrophobic drugs. Here, we studied how the properties of the silica framework of the mesoporous materials SBA-15, MCM-41, Al-MCM-41, and zeolites NaX, NaY, and HY affect the loading, stability, and distribution of encapsulated CLZ. Time-correlated single-photon counting (TCSPC) and fluorescence lifetime imaging microscopy (FLIM) experiments show the presence of neutral and protonated CLZ (1.3–3.8 ns) and weakly interacting aggregates (0.4–0.9 ns), along with H- and J-type aggregates (<0.1 ns). For the mesoporous and HY zeolite composites, the relative contribution to the overall emission spectra from H-type aggregates is low (<10%), while for the J-type aggregates it becomes higher (~30%). For NaX and NaY the former increased whereas the latter decreased. Although the CLZ@mesoporous composites show higher loading compared to the CLZ@zeolites ones, the behavior of CLZ is not uniform and its dynamics are more heterogeneous across different single mesoporous particles. These results may have implication in the design of silica-based drug carriers for better loading and release mechanisms of hydrophobic drugs.