Fourier solvers have become efficient tools to establish structure-property relations in heterogeneous materials. Introduced as an alternative to the Finite Element (FE) method, they are based on fixed-point solutions of the Lippmann-Schwinger type integral equation. Their computational efficiency results from handling the kernel of this equation by the Fast Fourier Transform (FFT). However, the kernel is derived from an auxiliary homogeneous linear problem, which renders the extension of FFT-based schemes to non-linear problems conceptually difficult. This paper aims to establish a link between FE-and FFT-based methods, in order to develop a solver applicable to general history-and time-dependent material models. For this purpose, we follow the standard steps of the FE method, starting from the weak form, proceeding to the Galerkin discretization and the numerical quadrature, up to the solution of non-linear equilibrium equations by an iterative Newton-Krylov solver. No auxiliary linear problem is thus needed. By analyzing a two-phase laminate with non-linear elastic, elasto-plastic, and visco-plastic phases, and by elasto-plastic simulations of a dual-phase steel microstructure, we demonstrate that the solver exhibits robust convergence. These results are achieved by re-using the non-linear FE technology, with the potential of further extensions beyond small-strain inelasticity considered in this paper.