In a recent publication, a procedure was developed which can be used to derive completely gauge invariant models from general Lagrangian densities with [Formula: see text] order of derivatives and [Formula: see text] rank of tensor potential. This procedure was then used to show that unique models follow for each order, namely classical electrodynamics for [Formula: see text] and linearized Gauss–Bonnet gravity for [Formula: see text]. In this paper, the nature of the connection between these two well-explored physical models is further investigated by means of an additional common property; a complete dual formulation. First, we give a review of Gauss–Bonnet gravity and the dual formulation of classical electrodynamics. The dual formulation of linearized Gauss–Bonnet gravity is then developed. It is shown that the dual formulation of linearized Gauss–Bonnet gravity is analogous to the homogenous half of Maxwell’s theory; both have equations of motion corresponding to the (second) Bianchi identity, built from the dual form of their respective field strength tensors. In order to have a dually symmetric counterpart analogous to the nonhomogenous half of Maxwell’s theory, the first invariant derived from the procedure in [Formula: see text] can be introduced. The complete gauge invariance of a model with respect to Noether’s first theorem, and not just the equation of motion, is a necessary condition for this dual formulation. We show that this result can be generalized to the higher spin gauge theories, where the spin-[Formula: see text] curvature tensors for all [Formula: see text] are the field strength tensors for each [Formula: see text]. These completely gauge invariant models correspond to the Maxwell-like higher spin gauge theories whose equations of motion have been well explored in the literature.