The objective of this study was to prepare and characterize physically crosslinked gel formulations of chitosan (CS)-graft-poly(N-isopropyl acrylamide) (PNIPAAm) and polyvinyl alcohol (PVA) for smart delivery of an antifungal drug, Voriconazole, for mucosal applications. For this purpose, cryogels of CS-g-PNIPAAm/PVA and CS/PVA were tested by means of texture profile analysis and rheology to determine optimal matrix properties for topical application. The ratio of 75/25 v/v % CS-g-PNIPAAm/PVA was selected to be used for formulation since it gave low compressibility and hardness (1.2 and 0.6 N) as well as high adhesion properties and non-Newtonian flow behavior. The cryogels and formulations were further characterized by means of FTIR spectroscopy, swelling behavior, texture analysis, scanning electron microscopy (SEM), thermal (differential scanning calorimetry (DSC) and TGA), and rheological behavior. The drug loading capacity and in vitro release profile of the drug, storage stability, and cytotoxicity tests were also performed for the gel formulation. The FTIR, DSC, and TGA results verified the successful formation of cryogels. Swelling studies revealed a pH-dependent swelling ability with a maximum swelling degree of 1200% in acid and 990% in phosphate buffer (pH = 7.4). Thermal studies showed that CS-g-PNIPAAm/PVA 75/25 had higher thermal stability proving the structural complexity of the polymer. The loading capacity of Voriconazole was found to be 70% (w/w). The in vitro release profiles of Voriconazole showed Fickian release behavior for CS-g-PNIPAAm/PVA 75/25 gel with an approximate delivery of 38% within 8 h, slower than matrices containing unmodified chitosan. The storage stability test exhibited that the gel formulation was still stable even after aging for two months. Moreover, the cell culture assays revealed a non-toxic character of the polymeric matrix. Overall results showed that the CS-g-PNIPAAm/PVA 75/25 hydrogel has the potential to be used as a smart polymeric vehicle for topical applications.