Interactive Analysis of Word Vector Embeddings

Abstract

Word vector embeddings are an emerging tool for natural language processing. They have proven beneficial for a wide variety of language processing tasks. Their utility stems from the ability to encode word relationships within the vector space. Applications range from components in natural language processing systems to tools for linguistic analysis in the study of language and literature. In many of these applications, interpreting embeddings and understanding the encoded grammatical and semantic relations between words is useful, but challenging. Visualization can aid in such interpretation of embeddings. In this paper, we examine the role for visualization in working with word vector embeddings. We provide a literature survey to catalogue the range of tasks where the embeddings are employed across a broad range of applications. Based on this survey, we identify key tasks and their characteristics. Then, we present visual interactive designs that address many of these tasks. The designs integrate into an exploration and analysis environment for embeddings. Finally, we provide example use cases for them and discuss domain user feedback.

Latent Space Cartography: Visual Analysis of Vector Space Embeddings

“…Interaxis [KCPE16] and Explainers [Gle13] allow users to define semantic axes and subsequently project data points onto the axes. Applying this idea to word embeddings, related work [HG18, BCZ∗16a, BCZ∗16b] has mapped a set of words to two user‐defined concept axes. We similarly enable users to construct a semantic axis from two opposing concepts, but unlike methods that simply layout two user‐defined axes, we provide additional projection strategies to ensure orthogonality and highlight variations.…”

“…Researchers have also developed interactive visual analysis tools for latent spaces [STN∗16, JSL∗17, HG18, LBT∗18, LNH∗18]. Some tools focus on a subset of tasks [STN∗16, LBT∗18] in word embeddings, which we extend and bring to a broader range of latent spaces.…”

“…Some tools focus on a subset of tasks [STN∗16, LBT∗18] in word embeddings, which we extend and bring to a broader range of latent spaces. Heimerl and Gleicher [HG18] catalog domain‐specific tasks for word embeddings and propose separate designs for each task. We similarly perform a task analysis to include more diverse venues, and instead of tackling each task separately, we build our system to support an integrated workflow.…”

“…Currently, most examples in the literature rely on static, ad‐hoc visualizations. Several interactive tools exist [LBT∗18, HG18, STN∗16,LNH∗18], but they either focus on a specific domain and a narrow set of tasks, or tackle each task separately without combining them into an integrated tool. We seek to guide users through a comprehensive workflow that supports tasks common to latent spaces across various input data types and learning algorithms.…”

A User‐based Visual Analytics Workflow for Exploratory Model Analysis

“…Gleicher noted that a simplified class of models could be used in a VA application to trade off some performance in exchange for a more explainable analysis [Gle13]. Many other systems and techniques purport to render various types of models interpretable, including deep learning models [LSC∗18, LSL∗17, SGPR18, YCN∗15, BJY∗18], topic models [WLS∗10], word embeddings [HG18], regression models [MP13], classification models [PBD∗10, RSG16, ACD∗15], and composite models for classification [LXL∗18]. While model explanation can be very useful in EMA, it does not help a user discover models, it only helps interpret them.…”