The comparative approach is a powerful way to explore the relationship between brain structure and cognitive function. Thus far the field has been dominated by the assumption that a bigger brain somehow means better cognition. Correlations between differences in brain size or neuron number between species and differences in specific cognitive abilities exist, but these correlations are very noisy. Extreme differences exist between clades in the relationship between either brain size or neuron number and specific cognitive abilities. This means that correlations become weaker, not stronger, as the taxonomic diversity of sampled groups increases. Cognition is the outcome of neural networks. Here we propose that considering plausible neural network models will advance our understanding of the complex relationships between neuron number and different aspects of cognition. Computational modelling of networks suggests that adding pathways, or layers, or changing patterns of connectivity in a network can all have different specific consequences for cognition. Consequently, models of computational architecture can help us hypothesise how and why differences in neuron number might be related to differences in cognition. As methods in connectomics continue to improve and more structural information on animal brains becomes available we are learning more about natural network structures in brains, and we can develop more biologically plausible models of cognitive architecture. Natural animal diversity then becomes a powerful resource to both test the assumptions of these models and explore hypotheses for how neural network structure and network size might delimit cognitive function.