Blockchain has been a vibrant technology in the past decade, with a wide variety of applications across different industrial sectors. The concept of blockchain has been widely recognized as an enabler for cryptocurrency-based decentralized payments, with two major decentralized payment systems such as Bitcoin and Ethereum. However, the global acceptance of blockchain as a cryptocurrency sums up significant challenges that hinder the fast adaptation of cryptocurrency as a payment service enabler. In this survey, we explore the advantages of blockchain and its technical capabilities beyond cryptocurrency. We focus on the technical potential to ensure trust, data governance, and automation of the financial application domain utilizing the fundamental security features of blockchain, including consensus, digital signatures, and transparency. The significant subcomponents of trust, data management, and automation in banking and financial systems are also identified and discussed, including how blockchain and smart contracts can achieve the anticipated features of each subcomponent through their technical capabilities. In addition, we shed light on the position of blockchain-based applications in key application sectors of the banking and financing domain with a mapping of technical features with the application domains. Thereafter, the applicability of blockchain-based applications is evaluated with relevant regulatory definitions. Finally, we discuss open research challenges and potential future works with the blockchain in the domain of financial systems.