The distinct features of ATPSs (aqueous two-phase systems) have made it possible to promote the extraction efficiency of biomolecules. The purpose of this study is to discover an appropriate nanoparticle to design an economical optimal separation process, and to understand the underlying molecular mechanism which allows the partitioning of vanillin as a phenolic compound using nanoparticle-based ATPSs. To this aim, the capabilities of several different nanoparticles were investigated as additives for boosting the partition coefficient of vanillin in two different ATPSs made up of polyethylene glycol and sodium sulfate/polyethylene glycol and dextran. Also, in an attempt to explain the salting-out effect, the NRTL (Non-random Two Liquid) thermodynamic model was applied. The impact of very small amounts of modified carbon nanotubes on the enhancement of the partition coefficient of vanillin in the ATPS consisting of the biocompatible polymer(s) and salt was quite remarkable. The results showed that the partition coefficient of vanillin grew by almost 127 percent compared to the system without nanoparticle. The molecular mechanism underlying the increase in the partition coefficient was interpreted by taking advantage of structural analyses.