Improving official statistics in emerging markets using machine learning and mobile phone data

Jahani, Eaman; Sundsøy, Pål; Bjelland, Johannes; Bengtsson, Linus; Pentland, Alex; Montjoye, Yves-Alexandre de

doi:10.1140/epjds/s13688-017-0099-3

Cited by 39 publications

(34 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This accuracy is close to the estimate of the upper bound of that of the Bayes optimal classifier, indicating that the performance is close to optimal. In contrast, the standard regression techniques performed poorly in predicting the numerical age, similar to what was observed in egocentric prediction studies [19]. The classifier performance is generally good among peers, with those in the Y, L and O age groups at 77% accuracy and above.…”

Section: Discussionsupporting

confidence: 71%

Link-centric analysis of variation by demographics in mobile phone communication patterns

et al. 2020

View full text Add to dashboard Cite

We present a link-centric approach to study variation in the mobile phone communication patterns of users. Unlike most previous research on call detail records that focused on the variation of phone usage across individual users, we examine how the calling and texting patterns obtained from call detail records vary among pairs of users and how these patterns are affected by the nature of relationships between users. To demonstrate this link-centric perspective, we extract factors that contribute to the variation in the mobile phone communication patterns and predict demographics-related quantities for pairs of users. The time of day and the channel of communication (calls or texts) are found to explain most of the variance among pairs that frequently call each other. Furthermore, we find that this variation can be used to predict the relationship between the pairs of users, as inferred from their age and gender, as well as the age of the younger user in a pair. By looking at the classifier performance across different age and gender groups, we gain insights into how communication patterns vary across different relationships.August 1, 2019 1/19 networks [23,24] have also been adopted. These studies have mainly relied on the egocentric perspective, working on the implicit assumption that an individual's traits determine his or her calling and texting patterns. Thus, when using an egocentric approach in predicting individual user information, all calls and texts made by the target individual are aggregated over all its neighbors, so the links in the target individual's egocentric network are not differentiated from each other. While the communication patterns have been shown to vary with age and gender at the individual level as discussed above, the communication patterns between two individuals have also been found to vary with the relationship between them [25,26]. However, while an egocentric perspective works well in studying variation at the individual level, it cannot, by design, provide much insight into variation among pairs. We hope to expand knowledge in this area by focusing on the links rather than the nodes in the mobile phone communication network, analyzing each link independently from others instead of aggregating them over the egocentric network. Such link-centric approach has not been explored much in relation to mobile phone data [27,28], especially in prediction tasks. In particular, only Herrera et al. [23] have used isolated link information as well as the demographic information of one user in the link to predict the age and gender of the other user. Still, as their aim was to predict individual user information, they did not touch on the connection between the communication patterns and the relationships between the pairs of users.It is important to note that while the age and gender can be categorically confirmed from the datasets, determining the relationships between any pair of users from the mobile phone data relies on inference. In particular, unless otherwise specified, we mean such in...

show abstract

Section: Discussionsupporting

confidence: 71%

Link-centric analysis of variation by demographics in mobile phone communication patterns

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Martinez et al used small size of reliable data set, with 10,000 users, and by using multiple algorithms (SVM, Random Forest and K-means) the accuracy obtained was 80% when the percentage of predicted instances was reduced [23]. In [1] bandicoot 1 tool is used to extract more than 1400 behavioral features, with different categories, and tested those features with different algorithms such as random forest, SVM, KNN, and the accuracy of the model was 79.7% at best for predicting the gender at developing countries as in South Asia.…”

Section: Related Workmentioning

confidence: 99%

“…

…”

mentioning

confidence: 99%

Predicting customer’s gender and age depending on mobile phone data

2019

View full text Add to dashboard Cite

IntroductionNowadays, the mobile phone is one of the fastest growing technologies in the developing world with global penetration rates reaching 90% [1]. This makes it a huge warehouse for customer's data. That is, every action taken by the customer (short message service (SMS), Call or Internet session) gets recorded within the telecom operator, in the so called (CDRs). There are many types of CDRs used mainly by telecom billing systems. CDR contains a lot of information, (type of event, who is involved in this event, datetime, cell identifier where this event has taken place). This raw data represents a valuable source for analyzing human and social behavior [2]. In the agricultural domain [3] mobile phone data is used to analyze mobility and seasonal activity patterns related to livelihood zones in Senegal, by creating mobility profiles for population and segmentation. While in energy domain [4] this data is used to analyze human activity, facilitate population growth estimation in rural areas and extrapolate electricity needs. In health sector [5,6] mobile phone data is used to study the relation between human mobility AbstractIn the age of data driven solution, the customer demographic attributes, such as gender and age, play a core role that may enable companies to enhance the offers of their services and target the right customer in the right time and place. In the marketing campaign, the companies want to target the real user of the GSM (global system for mobile communications), not the line owner. Where sometimes they may not be the same. This work proposes a method that predicts users' gender and age based on their behavior, services and contract information. We used call detail records (CDRs), customer relationship management (CRM) and billing information as a data source to analyze telecom customer behavior, and applied different types of machine learning algorithms to provide marketing campaigns with more accurate information about customer demographic attributes. This model is built using reliable data set of 18,000 users provided by SyriaTel Telecom Company, for training and testing. The model applied by using big data technology and achieved 85.6% accuracy in terms of user gender prediction and 65.5% of user age prediction. The main contribution of this work is the improvement in the accuracy in terms of user gender prediction and user age prediction based on mobile phone data and end-to-end solution that approaches customer data from multiple aspects in the telecom domain. which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

show abstract

“…The current state of the art in predicting demographics from mobile phone data is a recent paper by Jahani et al [10] which relies on a large number of handengineered features (1440) provided by the open-source bandicoot toolbox [16] and a carefully tuned SVM with a radial basis function kernel. The features used are divided into two categories (individual, spatial) and based on carefully engineered definitions such as how to group together calls and text messages into conversations or compute the churn rate of common locations.…”

Section: Related Workmentioning

confidence: 99%

Modeling the Temporal Nature of Human Behavior for Demographics Prediction

Felbo

Sundsøy

Pentland

et al. 2017

Machine Learning and Knowledge Discovery in Databases

Self Cite

View full text Add to dashboard Cite

Abstract. Mobile phone metadata is increasingly used for humanitarian purposes in developing countries as traditional data is scarce. Basic demographic information is however often absent from mobile phone datasets, limiting the operational impact of the datasets. For these reasons, there has been a growing interest in predicting demographic information from mobile phone metadata. Previous work focused on creating increasingly advanced features to be modeled with standard machine learning algorithms. We here instead model the raw mobile phone metadata directly using deep learning, exploiting the temporal nature of the patterns in the data. From high-level assumptions we design a data representation and convolutional network architecture for modeling patterns within a week. We then examine three strategies for aggregating patterns across weeks and show that our method reaches state-of-the-art accuracy on both age and gender prediction using only the temporal modality in mobile metadata. We finally validate our method on low activity users and evaluate the modeling assumptions.

show abstract

Improving official statistics in emerging markets using machine learning and mobile phone data

Cited by 39 publications

References 34 publications

Link-centric analysis of variation by demographics in mobile phone communication patterns

Link-centric analysis of variation by demographics in mobile phone communication patterns

Predicting customer’s gender and age depending on mobile phone data

Modeling the Temporal Nature of Human Behavior for Demographics Prediction

Contact Info

Product

Resources

About