Abstract-Tuning game difficulty prior to release requires careful consideration. Players can quickly lose interest in a game if it is too hard or too easy. Assessing how players will cope prior to release is often inaccurate. However, modern games can now collect sufficient data to perform large scale analysis postdeployment and update the product based on these insights.AI Factory Spades is currently the top rated Spades game in the Google Play store. In collaboration with the developers, we have collected gameplay data from 27 592 games and statistics regarding wins/losses for 99 866 games using Google Analytics. Using the data collected, this study analyses the difficulty and behaviour of an Information Set Monte Carlo Tree Search player we developed and deployed in the game previously [1].The methods of data collection and analysis presented in this study are generally applicable. The same workflow could be used to analyse the difficulty and typical player or opponent behaviour in any game. Furthermore, addressing issues of difficulty or non-human-like opponents post-deployment can positively affect player retention.