We explain the recent numerical successes obtained by Tao Xiang's group, who developed and applied Tensor Renormalization Group methods for the Ising model on square and cubic lattices, by the fact that their new truncation method sharply singles out a surprisingly small subspace of dimension two. We show that in the two-state approximation, their transformation can be handled analytically yielding a value 0.964 for the critical exponent ν much closer to the exact value 1 than 1.338 obtained in the Migdal-Kadanoff approximation. We propose two alternative blocking procedures that preserve the isotropy and improve the accuracy to ν = 0.987 and 0.993 respectively. We discuss applications to other classical lattice models, including models with fermions, and suggest that it could become a competitor for Monte Carlo methods suitable to calculate accurately critical exponents, take continuum limits and study near-conformal systems in arbitrarily large volumes.PACS numbers: 05.10. Cc,05.50.+q,11.10.Hi,64.60.De,75.10.Hk The Renormalization Group (RG) ideas have triggered considerable conceptual and numerical progress in many branches of physics [1,2]. However, the basic method to thin down the number of degrees of freedom in configuration space [3], often called "block spinning", has remained a formidable computational challenge for most classical lattice models (e. g., O(N ) spin models and lattice gauge theories). A few years ago, inspired by the so-called tensor network states [4,5] introduced in the context of the density matrix RG method [6,7], a Tensor RG (TRG) approach of two-dimensional (2D) classical lattice models was proposed [8]. Successful approximations [8][9][10] were found for the Ising model on honeycomb and triangular lattices.Very recently, the TRG method was successfully extended to the Ising model on square and cubic lattices by Tao Xiang's group [11]. There are two important ingredients in their calculations. First, their formulation allows an exact block spinning procedure which separates neatly the degrees of freedom inside the block, which are integrated over, from those kept to communicate with the neighboring blocks. As explained below, this provides a more systematic way to implement ideas initiated by Migdal [12] and Kadanoff [13] (abbreviated as MK hereafter). The indices of the tensors run over some finite vector space of "states" associated with finite volume link configurations. Second, they used a new method, based on higher order singular value decomposition, which selects in a very economical way the most important states that insure the communication among the blocks. Calculations using of the order of 20 states can be carried on a laptop computer. The excellent agreement found with the Onsager solution in 2D for arbitrarily large volume suggests that TRG-based methods could become competitors for conventional Monte Carlo methods.In this Letter, we show that the truncation method of Ref. [11] for the 2D Ising model sharply singles out a twodimensional subspace of states. Keeping...