Many long noncoding RNAs (lncRNAs) are unstable and rapidly degraded in the nucleus by the nuclear exosome. An exosome adaptor complex called NEXT (nuclear exosome targeting) functions to facilitate turnover of some of these lncRNAs. Here we show that knockdown of one NEXT subunit, Mtr4, but neither of the other two subunits, resulted in accumulation of two types of lncRNAs: prematurely terminated RNAs (ptRNAs) and upstream antisense RNAs (uaRNAs). This suggested a NEXT-independent Mtr4 function, and, consistent with this, we isolated a distinct complex containing Mtr4 and the zinc finger protein ZFC3H1. Strikingly, knockdown of either protein not only increased pt/uaRNA levels but also led to their accumulation in the cytoplasm. Furthermore, all pt/uaRNAs examined associated with active ribosomes, but, paradoxically, this correlated with a global reduction in heavy polysomes and overall repression of translation. Our findings highlight a critical role for Mtr4/ZFC3H1 in nuclear surveillance of naturally unstable lncRNAs to prevent their accumulation, transport to the cytoplasm, and resultant disruption of protein synthesis.