Deploying CommunityCommands: A Software Command Recommender System Case Study

Abstract

In 2009 we presented the idea of using collaborative filtering within a complex software application to help users learn new and relevant commands (Matejka et al. 2009). This project continued to evolve and we explored the design space of a contextual software command recommender system and completed a six-week user study (Li et al. 2011). We then expanded the scope of our project by implementing CommunityCommands, a fully functional and deployable recommender system. CommunityCommands was a publically available plug-in for Autodesk’s flagship software application AutoCAD. During a one-year period, the recommender system was used by more than 1100 users. In this article, we discuss how our practical system architecture was designed to leverage Autodesk’s existing Customer Involvement Program (CIP) data to deliver in-product contextual recommendations to end-users. We also present our system usage data and payoff, and provide an in-depth discussion of the challenges and design issues associated with developing and deploying the software command recommender system. Our work sets important groundwork for the future development of recommender systems within the domain of end-user software learning assistance.

Machine Learning for Learnability of MDD tools

“…Autodesk has implemented one such system for Autocad to improve the learnability of their tool by contextually proposing previously unseen commands. The company has then conducted a large study with more than 1000 users [27]. The authors of the study conclude that recommender systems are indeed useful and have a rich future in software applications.…”

“…The authors of the study conclude that recommender systems are indeed useful and have a rich future in software applications. Damevski [27] as well as Bullmer [17] have explicitly explored classification algorithms to predict developer's behavior, much as we do. They report accuracies between 20-60%, which are in general lower than the accuracies we achieve.…”