An important, potential application of image-based techniques is to create photo-realistic image-based environments for interactive walkthrough. However, existing image-based studies are based on different assumptions with different focuses. There is a lack of a general framework or architecture for evaluation and development of a practical image-based system. In this paper, we propose an architecture to unify different image-based methods. Based on the architecture, we propose an image-based system to support interactive walkthrough of scalable environments. In particular, we introduce the concept of angular range, which is useful for designing a scalable configuration, recovering geometric proxy as well as rendering. We also propose a new method to recover geometry information even from outdoor scenes and a new rendering method to address the problem of abrupt visual changes in a scalable environment.