Currently, millions of tons of textile waste from the garment and textile industries are generated worldwide each year. As a promising option in terms of sustainability, textile waste fibers could be used as internal reinforcement of cement-based composites by enhancing ductility and decreasing crack propagation. To this end, two extensive experimental programs were carried out, involving the use of either fractions of short random fibers at 6–10% by weight or nonwoven fabrics in 3–7 laminate layers in the textile waste-reinforcement of cement, and the mechanical and durability properties of the resulting composites were characterized. Flexural resistance in pre- and post-crack, toughness, and stiffness of the resulting composites were assessed in addition to unrestrained drying shrinkage testing. The results obtained from those programs were analyzed and compared to identify the optimal composite and potential applications. Based on the results of experimental analysis, the feasibility of using this textile waste composite as a potential construction material in nonstructural concrete structures such as facade cladding, raised floors, and pavements was confirmed. The optimal composite was proven to be the one reinforced with six layers of nonwoven fabric, with a flexural strength of 15.5 MPa and a toughness of 9.7 kJ/m2.