We present a unified exact tensor network approach to compute the ground state energy, identify the optimal configuration, and count the number of solutions for spin glasses. The method is based on tensor networks with the Tropical Algebra defined on the semiring of (R ∪ {−∞}, ⊕, ). Contracting the tropical tensor network gives the ground state energy; differentiating through the tensor network contraction gives the ground state configuration; mixing the tropical algebra and the usual algebra counts the ground state degeneracy. The approach brings together the concepts from graphical models, tensor networks, differentiable programming, and quantum circuit simulation, and easily utilizes computational power of graphical processing units (GPU). For applications, we compute the exact ground state energy of Ising spin glasses with random Gaussian couplings and fields on square lattice up to 1024 spins on a single GPU. Furthermore, we obtain exact ground state energy of ±J Ising spin glass on the chimera graph of D-Wave quantum annealer of 512 qubits in less than 100 seconds. Lastly, we investigate the exact value of the residual entropy of ±J spin glasses on the chimera graph which has not been reported before.