Transformers algorithms and deep learning architectures are increasingly spreading in different applications in several industries. One of the most well-known transformers widely used is ChatGPT. Transformers are designed to assist in problem solving by transforming sequence of inputs to a sequence of outputs. In this paper, transformer models will be studied for the purpose of integrating them to Business Intelligence (BI) architecture in Higher Education (HE) context. The purpose of integrating transformers is to minimise human interaction with data handling to benefit from time reduction and increase the quality of data. Quality Assurance (QA) in HE encompasses data gathering from several sources which requires data preparation to capture compliance with QA standards and measuring learning outcomes. This paper proposes an architecture for BI dashboards in HE with an integration with transformers models.