Numerous skeletal remains recovered in situ from the late early to middle Paleocene Takatika Grit of Chatham Island, New Zealand, are among the oldest known fossils attributed to the penguin clade (Aves, Sphenisciformes). They represent a new medium-sized taxon, for which we erect a new genus and species, and a second, notably larger form. These new penguins are analysed in a parsimony and Bayesian framework using an updated and revised phylogenetic matrix, based on morphological and molecular characters, and interpreted as among the most basal of known sphenisciforms, closely related to Waimanu. While sharing numerous characteristics with the earliest wing-propelled divers, the novel taxon records the oldest occurrence of the characteristic penguin tarsometatarsus morphology. These ancient Chatham Island representatives add to a growing number and increased morphological diversity of Paleocene penguins in the New Zealand region, suggesting an origin for the group there. With their addition to other Paleocene penguins, these taxa reveal that sphenisciforms rapidly diversified as non-volant piscivores in the southern oceans following the end-Cretaceous mass extinction. They also provide further evidence for the hypothesis that their origin predates the Paleocene. This implies that stem Sphenisciformes and their sister group, the Procellariiformes, both originated in, and so may be expected to occur in, the Late Cretaceous.