A novel quaternary CuMnSnSe diamondlike thermoelectric material was discovered recently based on the pseudocubic structure engineering. In this study, we show that introducing off-stoichiometry in CuMnSnSe effectively enhances its thermoelectric performance by simultaneously optimizing the carrier concentrations and suppressing the lattice thermal conductivity. A series of nonstoichiometric CuMnSnSe (δ = 0, 0.025, 0.05, 0.075, and 0.1) samples has been prepared by the melting-annealing method. The X-ray analysis and the scanning electron microscopy measurement show that all nonstoichiometric samples are phase pure. The Rietveld refinement demonstrates that substituting part of Mn by Cu well maintains the structure distortion parameter η close to 1, but it induces obvious local distortions inside the anion-centered tetrahedrons. Significantly improved carrier concentrations are observed in these nonstoichiometric CuMnSnSe samples, pushing the power factors to the theoretical maximal value predicted by the single parabolic model. Substituting part of Mn by Cu also reduces the lattice thermal conductivity, which is well interpreted by the Callaway model. Finally, a maximal thermoelectric dimensionless figure-of-merit zT around 0.60 at 800 K has been obtained in CuMnSnSe, which is about 33% higher than that in the CuMnSnSe matrix compound.