Mining Massive Fine-Grained Behavior Data to Improve Predictive Analytics

Abstract

Organizations increasingly have access to massive, fine-grained data on consumer behavior. Despite the hype over "big data," and the success of predictive analytics, only a few organizations have incorporated such finegrained data in a non-aggregated manner into their predictive analytics. This paper examines the use of massive, fine-grained data on consumer behavior-specifically payments to a very large set of particular merchants-to improve predictive models for targeted marketing. The paper details how using this different sort of data can substantially improve predictive performance, even in an application for which predictive analytics has been applied for years. One of the most striking results has important implications for managers considering the value of big data. Using a real-life data set of 21 million transactions by 1.2 million customers, as well as 289 other variables describing these customers, the results show that there is no appreciable improvement from moving to big data when using traditional structured data. However, in contrast, when using fine-grained behavior data, there continues to be substantial value to increasing the data size across the entire range of the analyses. This suggests that larger firms may have substantially more valuable data assets than smaller firms, when using their transaction data for targeted marketing.

Retail Credit Scoring using Fine-Grained Payment Data

“…In terms of lift it appears that adding the implied network score to the direct plus rating model does not lead to higher performance: when using 5 months of data, the lift of the full ensemble model and the direct plus rating model overlap. The results are in line with other studies that use relational learners on fine‐grained data: network data give a boost to the model lift (Martens et al ., ). In practical terms, this means that among the highest (worst) scores of the models that include payment data in a direct network there are more actual defaulters than among the highest scores of the traditional rating model.…”

“…Contrary to previous studies (Junqué de Fortuny et al ., 2013), we do not increase the number of clients, but the number of counterparties and known transactions per client. Nonetheless, we find the same conclusions: when working with fine‐grained data, bigger is better (Junqué de Fortuny et al ., 2013; Martens et al ., ).…”

“…The use of behavioural data has proven to be successful in other domains such as targeted advertising (Martens et al, 2016), fraud detection (Juqué de Fortung et al, 2014) and customer retention (Verbeke et al, 2014). The nature of these large behavioural data sets requires a modelling approach that is different from the traditional, structured data sets.…”

“…By creating both networks, we rely on the sociological concept of assortativity which states that people are more likely to form bonds with others who have similar characteristics such as values, beliefs and socio-economic status (McPherson et al, 2001). By creating a direct network, we build on the theory of assortativity and assume that people of similar creditworthiness tend to cluster (Martens et al, 2016).…”

“…The implied network is named a pseudosocial network: as in a true social network, strongly connected consumers demonstrate a strong similarity, at the very least in the particular merchants with whom they transact. It is a pseudosocial network because, by and large, the linked consumers probably have no true social relationship with one another (Martens et al, 2016).…”