We systematically examine the role of the self-assembly state of polymer-grafted silica nanoparticles (GNPs) on the spherulitic growth kinetics of a poly(ethylene oxide) (PEO) matrix. The nanoparticles (NPs) are functionalized with either unimodal or bimodal polymer brushes to systematically control their self-assembly within the semicrystalline polymer matrix. In the former case, we employ a poly(methyl methacrylate) (PMMA) brush (miscible with PEO), while the latter has a short dense polystyrene (PS carpet) brush, which is immiscible with the PEO matrix, and a long, sparse PMMA brush. The unimodal GNPs always yield well-dispersed NPs possibly because both the silica NP surface and the PMMA interact favorably with the PEO. The addition of the short PS carpet makes the interaction between the surface and the matrix PEO unfavorable. However, since the NPs also have grafted PMMA chains, they act akin to surfactants and provide access to a range of self-assembled structures. In all cases, the addition of NPs decreases the PEO spherulitic growth rates. Surprisingly, we find that the spatial dispersion of NPs does not change the secondary nucleation activation energy barrier of the PEO crystallization, such that the temperature dependence of spherulite growth kinetics is relatively unaffected by the NPs. Instead, the main effect of the NPs on spherulitic growth kinetics arises from variations of the melt's viscosity. Two apparently "universal" trends are foundbare NPs and large assemblies of GNPs appear to approximately follow the same dependence for the role of additives on polymer viscosity. On the other hand, all self-assembled NP structures which have at least one nanoscale dimension (well-dispersed NPs, one-dimensional strings or two-dimensional sheets) follow another general behavior, but one where the viscosity is much larger. Such unusual viscosity increases have been previously observed in the pioneering experiments of Composto and Winey, but there is currently no available theoretical description that can capture these changes and their effects on polymer crystallization.