Knowledge produced online often comes in the form of free-text labels, known as tags, with which users annotate the content they create, such as photos and videos. Increasingly, such content is also georeferenced, i.e., it is associated with geographic coordinates. The implicit relationships between tags and their locations can tell us much about how people conceptualize places and relations between them. However, extracting such knowledge from social annotations presents many challenges, since annotations are often ambiguous, noisy, uncertain and spatially inhomogeneous. We introduce a probabilistic framework for modeling georeferenced annotations and a method for learning model parameters from data. The framework is flexible and general, and can be used in a variety of applications that mine geospatial knowledge from user generated content. Specifically, we study two problems -extracting place semantics and predicting locations of photos from tags -and show that performance of our method is comparable to that of state-ofthe-art approaches. Moreover, we show that combining the two problems leads to a better performance on the location prediction task than baseline.