Part-of-speech (POS) tagging is a foundational task in natural language processing (NLP), which commonly leverages the learned representations of individual word and character encodings within those words. However, existing approaches often overlook the semantic richness of sub-word units like roots, stems, and affixes, especially in languages with complex morphology and limited resources. For this reason, this becomes a major limitation that leads to numerous unknown words and ambiguities in POS tagging task for agglutinative languages. In this work, we propose a novel approach that leverages deep representations of word prefixes and suffixes using character n-grams approximation method, augmenting features at both word and character levels. Additionally, we introduce a multi-head attention mechanism to attain contextual dependencies among words, effectively resolving POS tag ambiguities. Moreover, we create a customized dataset, named MultiPOS_ukg, for Uyghur, Uzbek, and Kyrgyz languages according to the uniform tag sets. Empirical evaluation on both the MultiPOS_ukg dataset and the METU Turkish Treebank dataset demonstrates a significant improvement in POS tagging accuracy. Specially, our approach achieves increases of up to 5.36%, 4.13%, and 2.1% across three languages. This improvement is achieved through the incorporation of affix-based word representation and multi-head attention, surpassing all other word and character-based models.