FinTech companies are challenging established financial institutions' dominance by offering the same products with a superior customer experience and delivering new features faster. The adoption of agile software development partially enables this competitive advantage. In response to this challenge, banks explore how they can improve their agile processes. LHV, a mid-sized bank, uses agile practices but faces the challenge of further improvement to stay competitive with FinTech companies.In this article, we explore how LHV can improve its agile software development process. We conduct a case study at LHV where we first derive eight change proposals based on a literature review and interviews. Then, we report on how LHV implemented the change proposals and their perceived impact. Our results stress the importance of taking a coherent approach to improving agile processes by considering both business units and operations involved in the product life-cycle. It is also necessary to align organizational structures to enable team autonomy by, for instance, decentralizing decision authority. Finally, it is beneficial to adapt agile practices to their context and have an IT architecture and technology supporting the agile approach.