One of the relevant text mining tasks is the document classification, where a useful content categorization control in many domains like content analyses, retrieval information, and the recommendation systems. In general, a set of process influence the classification system effectiveness, and the data representation has an essential impact on the text categorization as we will discover in this article. Hence, the paper's goal is to adjust the Paragraph Vector-Distributed Memory (PV-DM) as a variant of the current methods for neural text representation by comparing diverse neural parameters choices control the system complexity, e.g., epoch number, and vector size. Also, we employ a collection of classifiers subsequently combined using majority voting to show the impact of the neural PV-DM embedding on the binary business sentiment analysis, and multi labeled News data classification. The experiments prove that a suitable selection of the neural embedding characteristics enhances the hybrid machine learning model to 99% accuracy for a data type.