We analyze the early-time isotropic cosmology in the so-called Energy-Momentum Squared Gravity (EMSG). In this theory, a T µν T µν term is added to the Einstein-Hilbert action, which has been shown to replace the initial singularity by a regular bounce. We show that this is not the case, and the bouncing solution obtained does not describe our universe since it belongs to a different solution branch. The solution branch that corresponds to our universe, while non-singular, is geodesically incomplete. We analyze the conditions for having viable regular-bouncing solutions in a general class of theories that modify gravity by adding higher order matter terms. Applying these conditions on generalizations of EMSG that add a (T µν T µν ) n term to the action, we show that the case of n = 5/8 is the only one that can give a viable bouncing solution, while the n > 5/8 cases suffer from the same problem as EMSG, i.e. they give non-singular, geodesically incomplete solutions. Furthermore, we show that the 1/2 < n < 5/8 cases can provide a nonsingular initially de-Sitter solution. Finally, the expanding, geodesically incomplete branch of EMSG or its generalizations can be combined with its contracting counterpart using junction conditions to provide a (weakly) singular bouncing solution. We outline the junction conditions needed for this extension and provide the extended solution explicitly for EMSG. In this sense, EMSG replaces the standard early-time singularity by a singular bounce instead of a regular one.Recent cosmological observations provided us with a strong evidence for the accelerating expansion of the universe [1,2]. Trying to understand this acceleration in general relativity (GR) one is led to two possibilities: exotic matter content or a cosmological constant (Λ).Although these possibilities can fit the observational data, they do not provide us with a fundamental explanation of this acceleration. In addition to this large scale problem, GR predicts its own doom at small scales through the occurrence of spacetime singularities [3], which are expected to be cured in a full theory of quantum gravity (or at least an effective approximation of it). These issues, has led to a plethora of modified-gravity theories (see [4,5] for a review, and also [6] for a review on the recent observational constraints). In these theories, GR is seen as an effective field theory (of a more general gravitational theory) that might get corrections either at very large or very small scales.Some of these modified theories have been shown to replace the initial cosmological singularity that occurs in GR by a regular bounce (see [7,8] for a review). Along these efforts, energy-momentum-squared gravity (EMSG), as dubbed by its original authors, was proposed in [9]. This theory modifies gravity by adding a T µν T µν term to the Einstein-Hilbert Lagrangian; it is a special case of theories that have a Lagrangian of the form f (R, T µν T µν ) which were first studied in [10].Further efforts were conducted to study generalizations of EMSG a...