Risk-based authentication (RBA) aims to protect users against attacks involving stolen passwords. RBA monitors features during login, and requests re-authentication when feature values widely differ from previously observed ones. It is recommended by various national security organizations, and users perceive it more usable and equally secure than equivalent two-factor authentication. Despite that, RBA is still only used by very few online services. Reasons for this include a lack of validated open resources on RBA properties, implementation, and configuration. This effectively hinders the RBA research, development, and adoption progress. To close this gap, we provide the first long-term RBA analysis on a real-world large-scale online service. We collected feature data of 3.3 million users and 31.3 million login attempts over more than one year. Based on the data, we provide (i) studies on RBA’s real-world characteristics, and its configurations and enhancements to balance usability, security, and privacy, (ii) a machine learning based RBA parameter optimization method to support administrators finding an optimal configuration for their own use case scenario, (iii) an evaluation of the round-trip time feature’s potential to replace the IP address for enhanced user privacy, and (iv) a synthesized RBA data set to reproduce this research and to foster future RBA research. Our results provide insights on selecting an optimized RBA configuration so that users profit from RBA after just a few logins. The open data set enables researchers to study, test, and improve RBA for widespread deployment in the wild.