The development of low-cost and eco-friendly materials for the removal of pollutants from water is one of the main modern challenges. For this purpose, molecularly imprinted polymers were prepared under optimized conditions starting from chitosan (CS), chemically or ionically modified with glycidyl methacrylate (GMA) or itaconic acid (ITA), respectively. 2,4-Dichlorophenoxyacetic acid (2,4-D) was used as a template, obtaining the CS_GMA and CS_ITA series. The influence of the template concentration on the MIPs’ (molecularly imprinted polymers) morphology, thermal behaviour and swelling ability, as well as on the 2,4-D removal capacity, were analyzed. The amount of the template used for the imprinting, together with the different permeability of the matrices, were the key factors driving the analyte uptake process. Despite the good performance shown by the non-imprinted CS_GMA sample, the best results were obtained when CS_GMA was imprinted with the highest amount (5%) of template (CS_GMA_5). This system was also more efficient when consecutive adsorption experiments were carried out. In addition, CS_GMA_5 had a desorption efficiency of 90–100% when a low pesticide concentration was used. These findings suggest that the presence of imprinted cavities could be useful in improving the performance of sorbent materials making CS_GMA_5 a possible candidate for 2,4-D removal.