Capabilities and Limitations of Large Language Models

Abstract

The work is dedicated to the study of large language models (LLMs) and approaches to improving their efficiency in a new service. The rapid development of LLMs based on transformer architecture has opened up new possibilities in natural language processing and the automation of various tasks. However, fully utilizing the potential of these models requires a thorough approach and consideration of numerous factors. A review of the evolution of large language models was conducted, highlighting leading companies engaged in the research and development of efficient systems. The structure of these models and ways of representing internal knowledge were examined. Key approaches to training were described, including data collection, preprocessing, and selecting appropriate neural network architectures used in large language models. It was noted that the greatest breakthrough was achieved with the Transformer neural network, which is based on the attention mechanism. A comparison of popular transformer-based chatbots was presented, namely: ChatGPT, Claude AI, and Gemini AI. Their metrics, capabilities, and limitations were identified. The relevance of the topic lies in the rapid development of natural language processing technologies and the growing demand for large language models across various industries. The effective use of these models has tremendous potential to improve productivity and the quality of work with textual data. However, due to the complexity of the architecture and the large amounts of data required for training, selecting and configuring the optimal model for a specific task is a challenging process. As a result of the study, recommendations for developers were provided on the use of popular open-source models in the new service or integration with third-party programs. The characteristics of the models, their strengths, limitations, and certain caveats regarding trust in the generated results were indicated. Keywords: large language models, transformer architecture, neural networks, chatbot, content generation.