In the field of cultural heritage, the use of natural gels is rising for the application of active agents. Here, two natural polymers are assessed: agar, a pioneer hydrogel for conservation treatments, and chitosan, a rather novel and metal-binding gel. For chitosan, a state-of-the-art based formulation (CS–ItA–LCys) is evaluated as it was reported for silver-complexing properties. It is evaluated whether these polymers can withstand the addition of the chelating compound deferoxamine, which is a bacterial siderophore. This allows for the obtainment of completely bio-sourced gel systems. A Fourier-transformed (FT) infrared spectroscopy characterization is performed, completed with rheological measurements and Cryo-Scanning Electron Microscopy (cryo–SEM) to investigate the physico–chemical properties of the gels, as well as their interaction with deferoxamine. Both polymers are also tested for their inherent complexing ability on silver ions using FT–Raman spectroscopy. A multi-analytical comparison shows different microstructures, in particular, the presence of a thick membrane for chitosan and different mechanical behaviors, with agar being more brittle. Neither hydrogel seems affected by the addition of deferoxamine; this is shown by similar rheological behavior and molecular structures in the presence or absence of the chelator. The intrinsic abilities of the chitosan formulation to make silver complex are demonstrated with the observation of two peaks characteristic of Ag–S and Ag–O bonds. Agar and chitosan are both proven to be reliable gels to act as carriers for bio-based active agents. This paper confirms the potential asset of the chitosan formulation CS–ItA–LCys as a promising gel for the complexation of soluble silver.