Abstract-Aggregation services play an important role in the domain of Wireless Sensor Networks (WSNs) because they significantly reduce the number of required data transmissions, and improve energy efficiency on those networks. In most of the existing aggregation methods that have been developed based on the mathematical models or functions, the user of the WSN has not access to the original observations. In this paper, we propose an algorithm which let the base station access the observations by introducing a distributed method for computing the Principal Component Analysis (PCA). The proposed algorithm is based on transmission workload of the intermediate nodes. By using PCA, we aggregate the incoming packets of an intermediate node into one packet and as a result, an intermediate node merely sends a packet instead of relaying all the incoming packets. Consequently, we can achieve considerable reduction in data transmission. We have analyzed the performance of the proposed algorithm through numerical simulations. The experimental results show that our algorithm performs better than the existing state of the art PCA-based aggregation algorithms such as PCAg in terms of accuracy and efficiency.

The discrepancy between maximum likelihood estimation (MLE) and task measures such as BLEU score has been studied before for autoregressive neural machine translation (NMT) and resulted in alternative training algorithms (Ranzato et al., 2016;Norouzi et al., 2016;Shen et al., 2016;Wu et al., 2018). However, MLE training remains the de facto approach for autoregressive NMT because of its computational efficiency and stability. Despite this mismatch between the training objective and task measure, we notice that the samples drawn from an MLE-based trained NMT support the desired distribution -there are samples with much higher BLEU score comparing to the beam decoding output. To benefit from this observation, we train an energybased model to mimic the behavior of the task measure (i.e., the energy-based model assigns lower energy to samples with higher BLEU score), which is resulted in a re-ranking algorithm based on the samples drawn from NMT: energy-based re-ranking (EBR). We use both marginal energy models (over target sentence) and joint energy models (over both source and target sentences). Our EBR with the joint energy model consistently improves the performance of the Transformer-based NMT: +3.7 BLEU points on IWSLT'14 German-English, +3.37 BELU points on Sinhala-English, +1.4 BLEU points on WMT'16 English-German tasks.

This paper introduces rank-based training of structured prediction energy networks (SPENs). Our method samples from output structures using gradient descent and minimizes the ranking violation of the sampled structures with respect to a scalar scoring function defined using domain knowledge. We have successfully trained SPEN for citation field extraction without any labeled data instances, where the only source of supervision is a simple human-written scoring function. Such scoring functions are often easy to provide; the SPEN then furnishes an efficient structured prediction inference procedure.

Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural networks, which enables each neuron to learn a particular nonlinear activation function from a family of solutions to an ordinary differential equation. Specifically, each neuron may change its functional form during training based on the behavior of the other parts of the network. We show that using neurons with DEU activation functions results in a more compact network capable of achieving comparable, if not superior, performance when compared to much larger networks.

The biggest limitation of probabilistic graphical models is the complexity of inference, which is often intractable. An appealing alternative is to use tractable probabilistic models, such as arithmetic circuits (ACs) and sum-product networks (SPNs), in which marginal and conditional queries can be answered efficiently. In this paper, we present the first discriminative structure learning algorithm for ACs, DACLearn (Discriminative AC Learner), which optimizes conditional log-likelihood. Based on our experiments, DACLearn learns models that are more accurate and compact than other tractable generative and discriminative baselines.