Goal-driven Command Recommendations for Analysts

Abstract

Recent times have seen data analytics software applications become an integral part of the decision-making process of analysts. The users of these software applications generate a vast amount of unstructured log data. These logs contain clues to the user's goals, which traditional recommender systems may find difficult to model implicitly from the log data. With this assumption, we would like to assist the analytics process of a user through command recommendations. We categorize the commands into software and data categories based on their purpose to fulfill the task at hand. On the premise that the sequence of commands leading up to a data command is a good predictor of the latter, we design, develop, and validate various sequence modeling techniques. In this paper, we propose a framework to provide goal-driven data command recommendations to the user by leveraging unstructured logs. We use the log data of a web-based analytics software to train our neural network models and quantify their performance, in comparison to relevant and competitive baselines. We propose a custom loss function to tailor the recommended data commands according to the goal information provided exogenously. We also propose an evaluation metric that captures the degree of goal orientation of the recommendations. We demonstrate the promise of our approach by evaluating the models with the proposed metric and showcasing the robustness of our models in the case of adversarial examples, where the user activity is misaligned with selected goal, through offline evaluation. CCS CONCEPTS • Human-centered computing; • Computing methodologies → Neural networks; • Information systems → Recommender systems;

Understanding and generating realistic command distributions

“…Enfin dans le cadre de la prédiction de commandes (e.g., par des systèmes de recommendation) les dépendances entre commandes ont aussi été modélisées. On retrouve aussi des modèles de Markov d'ordre 1 [1], mais aussi des modèles plus complexes qui aggrègent plusieurs modèles de Markov (voir [15] pour plusieurs exemples), ou des réseaux de neurones profonds [33].…”

“…Pour caractériser les dépendances, on estime les probabilités de transitions pour toutes les paires de commandes. Une représentation de la matrice de transition associée à la première séquence du jeu de données est montrée sur le panneau de droite de la Figure 1, où les couleurs vont de jaune (0) à rouge (1). A noter que pour une meilleure visibilité 6 , l'ordre des commandes a été mélangé aléatoirement dans la matrice de transition, et dans tous le reste de l'article nous ferons la même chose.…”

Reinforced Approximate Exploratory Data Analysis

“…Similar to analyzing natural language texts to understand the topics, we can analyze query sequences to discover the user intents. Inspired by the previous work which uses topic modeling techniques to model goals that users carry out by executing a sequence of commands (Aggarwal et al 2020), we leverage topic modeling to discover the user intent, which is further used to derive the reward signal as detailed later in Section 4.3. Specifically, we use a bi-term model (BTM) (Yan et al 2013) for the identification of the intent.…”

“…Intent Analysis and Understanding: Several papers dealt with the concept of intents or goals for recommendations in process mining, web mining, education (Jiang et al 2019a,b) and analyze patterns in user behaviour from application log/clicks data (Aggarwal et al 2020;Dev and Liu 2017) or characterised SQL query logs to understand analysis behaviors (Jain et al 2016). Distinct from all these, we identify goal as a combination of queries and the corresponding results in a sequential decision making workflow.…”