Structure from Motion (SfM) plays a crucial role in unstructured capturing. While images are usually taken by perspective cameras, orthographic camera projections do not suffer from the foreshortening effect, that leads to varying capturing quality in image regions. Most contributions to orthographic image SfM assume a perspective setup with nearly infinite focal length. These assumptions lead to potentially sub-optimal camera pose estimation. Therefore, we propose a SfM pipeline that is optimized for orthographically projected images. For this, we estimate initial camera poses using the factorization method by Tomasi and Kanade. These poses are further refined by a specialized bundle adjustment implementation for orthographic projections. The proposed pipeline surpasses the precision of state-of-the-art work by an order of magnitude, while consuming considerably less computational resources.