A Statistical Language Modeling Approach to Online Deception Detection

Abstract

Online deception is disrupting our daily life, organizational process, and even national security. Existing approaches to online deception detection follow a traditional paradigm by using a set of cues as antecedents for deception detection, which may be hindered by ineffective cue identification. Motivated by the strength of statistical language models (SLMs) in capturing the dependency of words in text without explicit feature extraction, we developed SLMs to detect online deception. We also addressed the data-sparsity problem in building SLMs in general and in deception detection in specific using smoothing and vocabulary pruning techniques. The developed SLMs were evaluated empirically with diverse data sets. The results showed that the proposed SLM approach to deception detection outperformed a state-of-the-art text categorization method, as well as traditional feature-based methods.

Using Feature Selection and Classification Scheme for Automating Phishing Email Detection

“…We used 4 phishing dataset files as presented in Table 3. These phishing datasets have been used in phishing detection research including work by [3], [4], [5], [6], [9], [10], [12], and [16]. In order to provide non-phishing datasets, we used the SpamAssassin Project [17] from the easy ham directory.…”

Editorial

“…Features from the three approaches just introduced are used to train Naïve Bayes and Support Vector Machine classifiers, both of which have performed well in related work (Jindal and Liu, 2008;Mihalcea and Strapparava, 2009;Zhou et al, 2008).…”

“…Under (2), both the NB classifier used by Mihalcea and Strapparava (2009) and the language model classifier used by Zhou et al (2008) are equivalent. Thus, following Zhou et al (2008), we use the SRI Language Modeling Toolkit (Stolcke, 2002) to estimate individual language models, Pr( x | y = c), for truthful and deceptive opinions.…”

“…Thus, following Zhou et al (2008), we use the SRI Language Modeling Toolkit (Stolcke, 2002) to estimate individual language models, Pr( x | y = c), for truthful and deceptive opinions. We consider all three n-gram feature sets, namely UNIGRAMS, BIGRAMS + , and TRIGRAMS + , with corresponding language models smoothed using the interpolated Kneser-Ney method (Chen and Goodman, 1996).…”

A comparison of features for automatic deception detection in synchronous computer-mediated communication

“…The study aims to detect deception in text-based synchronous CMC by leveraging structural features extracted from online social networks [10], in addition to linguistic features [2], [11].…”