Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains a challenge.Here we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a dataefficient Graph Convolutional Network (GCN) algorithm PinSage, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model.We deploy PinSage at Pinterest and train it on 7.5 billion examples on a graph with 3 billion nodes representing pins and boards, and 18 billion edges. According to offline metrics, user studies and A/B tests, PinSage generates higher-quality recommendations than comparable deep learning and graph-based alternatives. To our knowledge, this is the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.

The SARS-CoV-2 B.1.617.2 (Delta) variant was first identified in the state of Maharashtra in late 2020 and spread throughout India, outcompeting pre-existing lineages including B.1.617.1 (Kappa) and B.1.1.7 (Alpha) 1 . In vitro, B.1.617.2 is 6-fold less sensitive to serum neutralising antibodies from recovered individuals, and 8-fold less sensitive to vaccine-elicited antibodies as compared to wild type (WT) Wuhan-1 bearing D614G. Serum neutralising titres against B.1.617.2 were lower in ChAdOx-1 versus BNT162b2 vaccinees. B.1.617.2 spike pseudotyped viruses exhibited compromised sensitivity to monoclonal antibodies against the receptor binding domain (RBD) and N-terminal domain (NTD). B.1.617.2 demonstrated higher replication efficiency in both airway organoid and human airway epithelial systems compared to B.1.1.7, associated with B.1.617.2 spike in a predominantly cleaved state compared to B.1.1.7. The B.1.617.2 spike protein was able to mediate highly efficient syncytium formation that was less sensitive to inhibition by neutralising antibody as compared to WT spike. Additionally we observed that B.1.617.2 had higher replication and spike mediated entry as compared to B.1.617.1, potentially explaining B.1.617.2 dominance. In an analysis of over 130 SARS-CoV-2 infected healthcare workers across three centres in India during a period of mixed lineage circulation, we observed reduced ChAdOx-1 vaccine effectiveness against B.1.617.2 relative to non-B.1.617.2, with the caveat of possible residual confounding. Compromised vaccine efficacy against the highly fit and immune evasive B.1.617.2 Delta variant warrants continued infection control measures in the post-vaccination era. India's first wave of SARS-CoV-2 infections in mid-2020 was relatively mild and was controlled by a nationwide lockdown. Since easing of restrictions, India has seen expansion in cases of COVID-19 since March

In recent years, graph neural networks (GNNs) have emerged as a powerful neural architecture to learn vector representations of nodes and graphs in a supervised, end-to-end fashion. Up to now, GNNs have only been evaluated empirically-showing promising results. The following work investigates GNNs from a theoretical point of view and relates them to the 1-dimensional Weisfeiler-Leman graph isomorphism heuristic (1-WL). We show that GNNs have the same expressiveness as the 1-WL in terms of distinguishing non-isomorphic (sub-)graphs. Hence, both algorithms also have the same shortcomings. Based on this, we propose a generalization of GNNs, so-called k-dimensional GNNs (k-GNNs), which can take higher-order graph structures at multiple scales into account. These higher-order structures play an essential role in the characterization of social networks and molecule graphs. Our experimental evaluation confirms our theoretical findings as well as confirms that higher-order information is useful in the task of graph classification and regression.completing the equivalence. Since the power of the 1-WL has been completely characterized, see, e.g., (Arvind et al. 2015;Kiefer, Schweitzer, and Selman 2015), we can transfer these results to the case of GNNs, showing that both approaches have the same shortcomings.Going further, we leverage these theoretical relationships to propose a generalization of GNNs, called k-GNNs, which are neural architectures based on the k-dimensional WL algorithm (k-WL), which are strictly more powerful than GNNs. The key insight in these higher-dimensional variants is that they perform message passing directly between subgraph structures, rather than individual nodes. This higher-order form of message passing can capture structural information that is not visible at the node-level.Graph kernels based on the k-WL have been proposed in the past (Morris, Kersting, and Mutzel 2017). However, a key advantage of implementing higher-order message passing in GNNs-which we demonstrate here-is that we can design hierarchical variants of k-GNNs, which combine graph representations learned at different granularities in an end-to-end trainable framework. Concretely, in the presented hierarchical approach the initial messages in a k-GNN are based on the output of lower-dimensional k -GNN (with k < k), which allows the model to effectively capture graph structures of varying granularity. Many real-world graphs inherit a hierarchical structure-e.g., in a social network we must model both the ego-networks around individual nodes, as well as the coarse-grained relationships between entire communities, see, e.g., (Newman 2003)-and our experimental results demonstrate that these hierarchical k-GNNs are able to consistently outperform traditional GNNs on a variety of graph classification and regression tasks. Across twelve graph regression tasks from the QM9 benchmark, we find that our hierarchical model reduces the mean absolute error by 54.45% on average. For graph classification, we find that our hierarchical models...

Understanding how words change their meanings over time is key to models of language and cultural evolution, but historical data on meaning is scarce, making theories hard to develop and test. Word embeddings show promise as a diachronic tool, but have not been carefully evaluated. We develop a robust methodology for quantifying semantic change by evaluating word embeddings (PPMI, SVD, word2vec) against known historical changes. We then use this methodology to reveal statistical laws of semantic evolution. Using six historical corpora spanning four languages and two centuries, we propose two quantitative laws of semantic change: (i) the law of conformity-the rate of semantic change scales with an inverse power-law of word frequency; (ii) the law of innovation-independent of frequency, words that are more polysemous have higher rates of semantic change.

Summary SARS-CoV-2 Spike protein is critical for virus infection via engagement of ACE2 1 , and is a major antibody target. Here we report chronic SARS-CoV-2 with reduced sensitivity to neutralising antibodies in an immune suppressed individual treated with convalescent plasma, generating whole genome ultradeep sequences over 23 time points spanning 101 days. Little change was observed in the overall viral population structure following two courses of remdesivir over the first 57 days. However, following convalescent plasma therapy we observed large, dynamic virus population shifts, with the emergence of a dominant viral strain bearing D796H in S2 and ΔH69/ΔV70 in the S1 N-terminal domain NTD of the Spike protein. As passively transferred serum antibodies diminished, viruses with the escape genotype diminished in frequency, before returning during a final, unsuccessful course of convalescent plasma. In vitro , the Spike escape double mutant bearing ΔH69/ΔV70 and D796H conferred modestly decreased sensitivity to convalescent plasma, whilst maintaining infectivity similar to wild type. D796H appeared to be the main contributor to decreased susceptibility but incurred an infectivity defect. The ΔH69/ΔV70 single mutant had two-fold higher infectivity compared to wild type, possibly compensating for the reduced infectivity of D796H. These data reveal strong selection on SARS-CoV-2 during convalescent plasma therapy associated with emergence of viral variants with evidence of reduced susceptibility to neutralising antibodies.