Better Word Embeddings by Disentangling Contextual n-Gram Information

Gupta, Prakhar; Pagliardini, Matteo; Jäggi, Martin

doi:10.48550/arxiv.1904.05033

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A straightforward way of using Word2Vec to represent sentences is by averaging word vectors [12,70]. Furthermore, several Word2Vec extensions optimize and predispose the original method directly for sentence embedding [26,42,16]. A more substantial modification to the original Word2Vec model is Doc2Vec, which introduces paragraph vectors [31].…”

Section: Sentence Embeddingmentioning

confidence: 99%

Text to Time Series Representations: Towards Interpretable Predictive Models

Poggioli,

Spinnato,

Guidotti

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Natural Language Processing (NLP) and time series analysis are two domains of research that have seen a surge of interest in recent years. At the core of NLP are text representation techniques that convert text into machine-readable input. These techniques have proven remarkably accurate, especially with state-of-the-art transformer models. However, they inhibit human interpretation since they generate an implicit vector representation that embeds the text as a whole. In contrast, time series analysis techniques are inherently designed to extract meaningful information from time-dependent data while preserving the local features. One prominent time series classification approach uses shapelets to identify representative sequences of the target variable. This approach classifies time series based on their proximity to shapelets, distinguishing it from time-independent methods that rely on global time series statistics. In the context of time series, KNN classification has also been adapted with distance metrics like DTW that detect local patterns between time series even if they do not occur simultaneously. This study proposes a novel approach for text-to-time series representations that preserve the text's sequential structure. Specifically, we treat text as a sequence of sentences and map it using feature extraction techniques derived from sentence embedding, sentiment analysis, and custom linguistic features. We explore pooling methods to convert multivariate time series into univariate ones enhancing the identification of unique one-dimensional patterns among text-time series. We propose adapting the principal component analysis technique (PCA) to reduce sentence timestamps' global feature vector space to one dimension. By reducing the dimensionality, we can represent the text-time series as a one-dimensional signal that can be more easily analyzed and interpreted. We test the resulting transformation from several datasets consisting mainly of song lyrics, performing classification tasks. We compare feature-based approaches that use global statistics from multivariate time series with KNN and shapelet-based approaches on the resulting univariate time series. Despite the pooling approximation, shapelet-based methods perform similarly to feature-based ones, although with variation in results depending on the text domain, feature type, and dimensionality reduction contingencies. Finally, we demonstrate how to extract interpretable features from the original multivariate shapelets and relocate the shapelets in the source text.

show abstract

Section: Sentence Embeddingmentioning

confidence: 99%

Text to Time Series Representations: Towards Interpretable Predictive Models

Poggioli,

Spinnato,

Guidotti

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…For a given time window, we create a network whose nodes are concepts (aggregated keywords) present in articles published during that time window. Keywords are aggregated into "concepts" using embeddings from a Sent2Vec model (Gupta, Pagliardini, and Jaggi 2019); the simple average of token-level embeddings is used for keywords composed of multiple tokens. The Sent2Vec model is trained on a corpus of articles similar in domain to the domain that we analyze during the final scoring (e.g.…”

Section: Construction Of Networkmentioning

confidence: 99%

Identifying Early Warning Signals from News Using Network Community Detection

Vine

Boxer

Dinani

et al. 2022

AAAI

View full text Add to dashboard Cite

The paper addresses the challenge of accelerating identification of changes in risk drivers in the insurance industry. Specifically, the work presents a method to identify significant news events ("signals") from batches of news data to inform Life & Health insurance decisions. Signals are defined as events that are relevant to a tracked risk driver, widely discussed in multiple news outlets, contain novel information and affect stakeholders. The method converts unstructured data (news articles) into a sequence of keywords by employing a linguistic knowledge graph-based model. Then, for each time window, the method forms a graph with extracted keywords as nodes and draws weighted edges based on keyword co-occurrences in articles. Lastly, events are derived in an unsupervised way as graph communities and scored for the requirements of a signal: relevance, novelty and virality. The methodology is illustrated for a Life & Health topic using news articles from Dow Jones DNA proprietary data set, and assessed against baselines on a publicly available news data set. The method is implemented as an analytics engine in Early Warning System deployed at Swiss Re for the last 1.5 years to extract relevant events from live news data. We present the system's architectural design in production and discuss its use and impact.

show abstract

Better Word Embeddings by Disentangling Contextual n-Gram Information

Cited by 2 publications

References 0 publications

Text to Time Series Representations: Towards Interpretable Predictive Models

Text to Time Series Representations: Towards Interpretable Predictive Models

Identifying Early Warning Signals from News Using Network Community Detection

Contact Info

Product

Resources

About