Pre-trained Transformer-based models have achieved state-of-the-art performance for various Natural Language Processing (NLP) tasks. However, these models often have billions of parameters, and thus are too resource- hungry and computation-intensive to suit low- capability devices or applications with strict latency requirements. One potential remedy for this is model compression, which has attracted considerable research attention. Here, we summarize the research in compressing Transformers, focusing on the especially popular BERT model. In particular, we survey the state of the art in compression for BERT, we clarify the current best practices for compressing large-scale Transformer models, and we provide insights into the workings of various methods. Our categorization and analysis also shed light on promising future research directions for achieving lightweight, accurate, and generic NLP models.

Due to recent cyber attacks on various cyberphysical systems (CPSes), traditional isolation based security schemes in the critical systems are insufficient to deal with the smart adversaries in CPSes with advanced information and communication technologies (ICTs). In this paper, we develop real-time assessment and mitigation of an attack's impact as a system's built-in mechanisms. We study a general class of attacks, which we call time delay attack, that delays the transmissions of control data packets in the CPS control loops. Based on a joint stability-safety criterion, we propose the attack impact assessment consisting of (i) a machine learning (ML) based safety classification, and (ii) a tandem stability-safety classification that exploits a basic relationship between stability and safety, namely that an unstable system must be unsafe whereas a stable system may not be safe. In this assessment approach, the ML addresses a state explosion problem in the safety classification, whereas the tandem structure reduces false negatives in detecting unsafety arising from imperfect ML. We apply our approach to assess the impact of the attack on power grid automatic generation control, and accordingly develop a two-tiered mitigation that tunes the control gain automatically to restore safety where necessary and shed load only if the tuning is insufficient. We also apply our attack impact assessment approach to a thermal power plant control system consisting of two PID control loops. A mitigation approach by tuning the PID controller is also proposed. Extensive simulations based on a 37-bus system model and a thermal power plant control system are conducted to evaluate the effectiveness of our assessment and mitigation approaches.

No abstract

The Next Generation Sequencing Technique (NGS) has provided affordable and fast method for generating genetic data. Generation of whole Genome Sequence and extract relevant information from this data is still a computationally expensive process. In this paper we demonstrate a novel approach for local alignment of DNA reads with respect to reference genome. For this process we have used Skip-gram model for creating encoding(Nucl2Vec) and k-nearest neighbor for the alignment. With our new approach we have reduced computation cost for local alignment , while achieving accuracy comparable to existing defacto standard BWA-MEM tool.