Fiber–matrix interfacial adhesion is one of the key factors governing the final properties of natural fiber-based polymer composites. In this work, four extrusion reactive agents were tested as potential compatibilizers in polyhydroxylbutyrate (PHB)/cellulose composites: dicumyl peroxide (DCP), hexamethylene diisocyanate (HMDI), resorcinol diglycidyl ether (RDGE), and triglycidyl isocyanurate (TGIC). The influence of the fibers and the different reactive agents on the mechanical properties, physical aging, and crystallization behavior were assessed. To evaluate the compatibilization effectiveness of each reactive agent, highly purified commercial cellulose fibers (TC90) were used as reference filler. Then, the influence of fiber purity on the compatibilization effect of the reactive agent HMDI was evaluated using untreated (U_RH) and chemically purified (T_RH) rice husk fibers, comparing the results with the ones using TC90 fibers. The results show that reactive agents interact with the polymer matrix at different levels, but all compositions showed a drastic embrittlement due to the aging of PHB. No clear compatibilization effect was found using DCP, RDGE, or TGIC reactive agents. On the other hand, the fiber–polymer interfacial adhesion was enhanced with HMDI. The purity of the fiber played an important role in the effectiveness of HMDI as a compatibilizer, since composites with highly purified fibers showed the greatest improvements in tensile strength and the most favorable morphology. None of the reactive agents negatively affected the compostability of PHB. Finally, thermoformed trays with good mold reproducibility were successfully obtained for PHB/T_RH/HMDI composition.