Attention mechanisms have led to many breakthroughs in sequential data modeling but have yet to be incorporated into any generative algorithms for molecular design. Here we explore the impact of...

This article empirically examines the impact of globalization on the health status of countries by using panel data. Unlike previous studies, it has attempted to use three different dimensions of globalization and estimate their impact on health status measured by infant mortality rate and life expectancy. It also introduces an initial level of development status as an explanatory variable and found that it has an important role. The fixed effects panel data analysis shows that globalization has a positive impact on the health indicators. Out of the three dimensions of globalization, namely, economic, social and political, the first one has the highest influence on health for the less developed countries. However, as one moves up the ladder of development, social dimension becomes more important. Moreover, the pace of improvement in health indicators is faster in developed countries, indicating a divergence between the developed and the underdeveloped world.

<div> <div> <div> <p>We explore the impact of adding attention to generative VAE models for molecular design. Four model types are compared: a simple recurrent VAE (RNN), a recurrent VAE with an added attention layer (RNNAttn), a transformer VAE (TransVAE) and the previous state-of-the-art (MosesVAE). The models are assessed based on their effect on the organization of the latent space (i.e. latent memory) and their ability to generate samples that are valid and novel. Additionally, the Shannon information entropy is used to measure the complexity of the latent memory in an information bottleneck theoretical framework and we define a novel metric to assess the extent to which models explore chemical phase space. All three models are trained on millions of molecules from either the ZINC or PubChem datasets. We find that both RNNAttn and TransVAE models perform substantially better when tasked with accurately reconstructing input SMILES strings than the MosesVAE or RNN models, particularly for larger molecules up to ~700 Da. The TransVAE learns a complex “molecular grammar” that includes detailed molecular substructures and high-level structural and atomic relationships. The RNNAttn models learn the most efficient compression of the input data while still maintaining good performance. The complexity of the compressed representation learned by each model type increases in the order of MosesVAE < RNNAttn < RNN < TransVAE. We find that there is an unavoidable tradeoff between model exploration and validity that is a function of the complexity of the latent memory. However, novel sampling schemes may be used that optimize this tradeoff and allow us to utilize the information-dense representations learned by the transformer in spite of their complexity. </p> </div> </div> </div>