Abstract. The gametocyte-specific factor 1 (GTSF1) gene participates in DNA methylation and retrotransposon activation in germ cells, particularly during cell proliferation. The present study aimed to assess the level of GTSF1 gene expression in liver cancer tumor tissues, and its role in human hepatoma cell lines in vitro and in a nude mouse model in vivo. GTSF1 gene expression was detected in liver cancer tumor tissues, compared with in healthy controls, via reverse transcription quantitative polymerase chain reaction. An adeno-associated virus vector was used to study tumor stem cell proliferation in vivo. A plasmid expressing GTSF1 was constructed and transfected into various human hepatoma cell lines, in order to analyze the cellular proliferation and apoptosis of liver cancer cells using small interfering (si)RNAs in vitro. In the present study, GTSF1 gene expression was detected in 18/24 (75.0%) liver cancer tumor tissues from patients with hepatocellular carcinoma (HCC), and elevated GTSF1 expression was identified in the tissue of one of 32 healthy control samples (3.13%; P<0.05). Notably, the GTSF1 gene was expressed at a higher frequency in AFP-positive HCC samples (14/16, 87.50%) compared with in AFP-negative HCC samples (4/8, 50.0%; P=0.129). In addition, there was no statistical significance between GTSF1 expression in non-HBV-infected (71.42%) and HBV-infected HCC specimens (76.47%), as determined by a χ 2 test (P=0.921). It was demonstrated that GTSF1 significantly increased the tumorigenicity of Ad-shNC-transfected (GTSF1-positive) HepG2 cells in the nude mouse xenograft model, whereas the sizes and weights of the tumors in the GTSF1-negative group were dercreased in comparison with the GTSF1-positive group (P<0.05). Reduced levels of GTSF1 mRNA, along with fewer and smaller colonies, were identified in two groups of human liver cancer cells treated with with GTSF1-targeting siRNA, when compared with cells without GTSF1 mRNA interference (P<0.05). In summary, the present study elucidated the GTSF1 mRNA expression pattern in liver cancer, and investigated the potential role of GTSF1 in tumorigenesis. The data suggest an important role for the GTSF1 gene in the molecular etiology of hepatocarcinogenesis, and indicate a potential application of GTSF1 mRNA expression in liver cancer diagnosis and therapy.