The Cassini and Juno probes have revealed large coherent cyclonic vortices in the polar regions of Saturn and Jupiter, a dramatic contrast from the east-west banded jet structure seen at lower latitudes. Debate has centered on whether the jets are shallow, or extend to greater depths in the planetary envelope. Recent experiments and observations have demonstrated the relevance of deep convection models to a successful explanation of jet structure, and cyclonic coherent vortices away from the polar regions have been simulated recently including an additional stratified shallow layer. Here we present new convective models able to produce long-lived polar vortices. Using simulation parameters relevant for giant planet atmospheres we find flow regimes of geostrophic turbulence (GT) in agreement with rotating convection theory. The formation of large scale coherent structures occurs via three-dimensional upscale energy transfers. Our simulations generate polar characteristics qualitatively similar to those seen by Juno and Cassini: they match the structure of cyclonic vortices seen on Jupiter; or can account for the existence of a strong polar vortex extending downwards to lower latitudes with a marked spiral morphology, and the hexagonal pattern seen on Saturn. Our findings indicate that these vortices can be generated deep in the planetary interior. A transition differentiating these two polar flows regimes is described, interpreted in terms of force balances and compared with shallow atmospheric models characterising polar vortex dynamics in giant planets. In addition, heat transport properties are investigated, confirming recent scaling laws obtained with reduced models of GT.