Smart city middleware serves as a foundational tool in the evolution of urban digitalization, acting as an intermediary software layer that simplifies the development, deployment, and management of applications tailored for smart urban environments. However, the development of effective middleware for smart cities is challenging. The present research embarks on a comprehensive exploration of the smart city middleware landscape, unraveling the intricacies of its development and the challenges faced therein. Rooted in the assessment of 20 distinct middleware solutions, our study highlights the pivotal technologies, features and functionalities that are imperative for a middleware to effectively support a city's digital transformation. The functional and non-functional requirements form the nucleus of our evaluation. We also explore the architectural styles pivotal to middleware development and the programming paradigms shaping smart city application development. Our study highlights challenges in using middleware for smart city applications, such as interoperability, scalability, security amidst big data, context management, reliability, quality of service, energy efficiency, and compliance with technological standards and regulations. Based on the detailed analysis, we propose a conceptual framework for smart city middleware, shaped by the challenges and requirements identified in existing literature and middleware solutions. This framework is designed to reflect the diverse demands and complexities of urban digital transformation, and guide smart city middleware development accordingly. As a result, this research stands as a reference study for software developers, urban planners, and researchers, outlining the current state and future directions in the domain of smart city middleware.