Recent measures introduced by Australian governments to improve the visual-quality of the built environment have been criticised as being too prescriptive. Two opposing models of people's aesthetic response to buildings have been proposed in the research literature. The preference-for-difference model has gained more support than the preference-for-prototype model but is at odds with current government policies. Both models are based on studies using photographs of buildings in laboratory settings, but have not been replicated through people's actual experience of buildings in the real-world. This paper reports on a field study conducted to examine whether the preference-for-difference model could explain the public's perception and aesthetic response to a building controversial for its contemporary architecture. Fifty people were interviewed in and immediately around Federation Square, Melbourne and asked to make aesthetic judgements of the Square. The public collectively found the building to be interesting, moderately unusual, and pleasing. The results supported the preference-for-difference model of aesthetic response. It is suggested that more field studies examining aesthetic response to buildings should be conducted. More importantly, and perhaps surprisingly, these findings suggest that to meet public aesthetic views, government policy should encourage diversity and innovation in design and aesthetics.