Refactoring is one of the techniques mostly employed by software developers to improve the quality attributes of their systems. However, little has been done to investigate how refactoring operations specifically aimed at improving the internal structure of software can impact its security. Refactoring usually entails different code change operations including the decomposition of classes, methods, and the reallocation of code elements. While this refinement aims to improve the internal design of a system, it might inadvertently disperse security-critical code elements throughout the codebase. Consequently, such dispersion could affect the software attack surface. To this end, this paper presents an empirical study of 30 open-source software systems developed in Python, C, and Javascript. The study scrutinized two subsequent versions of each subject application to uncover the refactoring operations applied and the trend of the software attack surface. Specifically, the study focused on the injection or removal of bugs, code smells and other vulnerabilities aiming to discern the impact of refactorings on the software attack surface. Data was collected using wellknown tools, namely SonarQube, RefDiff, and PyReff. The findings suggest that refactorings can have multiple impacts (i.e., positive, negative, or neutral) on bugs, code smells, and vulnerabilities. The findings further confirm that developers must be aware of the combination or sequence of refactoring operations that can improve software quality without compromising its security.