This paper investigates the performance of an amplify-and-forward (AF) two-way relaying system with antenna correlation. The system consists of two multiple-antenna sources, which exchange information via the aid of a single-antenna relay. In particular, we derive the exact outage probability expression. Furthermore, we provide a simple, tight closed-form lower bound for the outage probability. Based on the lower bound, we obtain the closed-form asymptotic outage probability and the average symbol error rate expressions at high signal-to-noise ratio (SNR), which reveal the system's diversity order and coding gain with antenna correlation. To investigate the system's throughput performance with antenna correlation, we also derive a closed-form lower bound for the average sum-rate, which is quite tight from medium to high SNR regime. The analytical results readily enable us to obtain insight into the effect of antenna correlation on the system's performance. Extensive Monte Carlo simulations are conducted to verify the analytical results.