Tensor network states, and in particular projected entangled pair states (PEPS), suggest an innovative approach for the study of lattice gauge theories, both from a pure theoretic point of view, and as a tool for the analysis of the recent proposals for quantum simulations of lattice gauge theories. In this paper we present a framework for describing locally gauge invariant states on lattices using PEPS. The PEPS constructed hereby shall include both bosonic and fermionic states, suitable for all combinations of matter and gauge fields in lattice gauge theories defined by either finite or compact Lie groups.This last step has been undertaken for two-dimensional tensor networks in the recent works [46,48,55]. In particular, Tagliacozzo et al [46] introduced a formalism for building bosonic PEPS describing pure lattice gauge theories with arbitrary groups and adopted a truncation scheme for the representations which we will extend in this work to include matter and fermionic PEPS (fPEPS) as well; Haegeman et al [48] developed, instead, a general, conceptual formalism to build PEPS which describe both gauge fields and bosonic matter through a gauging map of injective PEPS. Finally, in [55], the authors presented an explicit analytical and numerical construction for the specific case of a truncated U(1) gauge-invariant state including both fermionic matter and bosonic gauge fields.In this work, we develop a more general framework for the construction of PEPS with local gauge invariance for lattice gauge theories whose gauge group G is either a compact Lie, or a finite group. We aim at unifying all the elements required for complete descriptions of lattice gauge theories: in particular we focus on gauge invariant states which include both matter and gauge fields and we propose a constructive approach to define tensor network states where the matter can be either bosonic or fermionic and is associated to an arbitrary representation of the gauge group. A special attention will indeed be devoted to the construction of fPEPS because of their more immediate relevance in the study of models which are closer to the usual particle physics theories.Throughout this paper we adopt the mathematical stucture discussed in [57] to describe the physical components of the PEPS and, in particular, the Hilbert spaces used to describe the matter and the gauge field degrees of freedom. We will show that such description, based on the representations of the gauge group, can be naturally extended to the virtual states constituting the links of the tensor network. Furthermore, to make this work self-contained, we will summarize the main elements of the construction presented in [57] without assuming that the reader is acquainted with PEPS or lattice gauge theories; the required PEPS details are reviewed and constructed along the paper, as well as their relations with lattice gauge theory.