Online template induction for machine-generated emails

Abstract

In emails, information abounds. Whether it be a bill reminder, a hotel confirmation, or a shipping notification, our emails contain useful bits of information that enable a number of applications. Most of this email traffic is machine-generated, sent from a business to a human. These business-to-consumer emails are typically instantiated from a set of email templates, and discovering these templates is a key step in enabling a variety of intelligent experiences. Existing email information extraction systems typically separate information extraction into two steps: an offline template discovery process (called template induction) that is periodically run on a sample of emails, and an online email annotation process that applies discovered templates to emails as they arrive. Since information extraction requires an email's template to be known, any delay in discovering a newly created template causes missed extractions, lowering the overall extraction coverage. In this paper, we present a novel system called Crusher that discovers templates completely online, reducing template discovery delay from a week (for the existing MapReduce-based batch system) to minutes. Furthermore, Crusher has a resource consumption footprint that is significantly smaller than the existing batch system. We also report on the surprising lesson we learned that conventional stream processing systems do not present a good framework on which to build Crusher. Crusher delivers an order of magnitude more throughput than a prototype built using a stream processing engine. We hope that these lessons help designers of stream processing systems accommodate a broader range of applications like online template induction in the future.

Classifying Emails into Human vs Machine Category

“…There are a lot of well-known applications on mail area, including spam detection (Kumaresan, Saravanakumar, and Balamurugan 2019;Douzi et al 2020; Mohammadzadeh and Gharehchopogh 2021) user's future action prediction (Di Castro et al 2016), email threading (Ailon et al 2013) and information extraction (Agarwal and Singh 2018;Di Castro et al 2018;Sheng et al 2018). Recently, research community has made more interesting applications, such as cyber security events detection (Vinayakumar et al 2019), commitment detection (Azarbonyad, Sim, and White 2019), intent detection (Shu et al 2020) and online template induction (Whittaker et al 2019). Most of these applications depend on training high-quality classifiers on emails.…”