To explore the effect of polymer structure on their self-assembled aggregates and their unique characteristics, this study was devoted to developing a series of amphiphilic block and random phenylboronic acid-based glycopolymers by RAFT polymerization. The amphiphilic glycopolymers were successfully self-assembled into spherically shaped nanoparticles with narrow size distribution in aqueous solution. For block and random copolymers with similar monomer compositions, block copolymer nanoparticles exhibited a more regular transmittance change with the increasing glucose level, while a more evident variation of size and quicker decreasing tendency in I/I0 behavior in different glucose media were observed for random copolymer nanoparticles. Cell viability of all the polymer nanoparticles investigated by MTT assay was higher than 80%, indicating that both block and random copolymers had good cytocompatibility. Insulin could be encapsulated into both nanoparticles, and insulin release rate for random glycopolymer was slightly quicker than that for the block ones. We speculate that different chain conformations between block and random glycopolymers play an important role in self-assembled nanoaggregates and underlying glucose-sensitive behavior.