Existed pre-training methods either focus on single-modal tasks or multi-modal tasks, and cannot effectively adapt to each other. They can only utilize single-modal data (i.e., text or image) or limited multi-modal data (i.e., image-text pairs). In this work, we propose a UNIfied-MOdal pre-training architecture, namely UNIMO, which can effectively adapt to both single-modal and multi-modal understanding and generation tasks. Large scale of free text corpus and image collections are utilized to improve the capability of visual and textual understanding, and crossmodal contrastive learning (CMCL) is leveraged to align the textual and visual information into a unified semantic space, over a corpus of image-text pairs augmented with related images and texts. With the help of rich non-paired single-modal data, our model is able to learn more generalizable representations, by allowing textual knowledge and visual knowledge to enhance each other in the unified semantic space. The experimental results show that UNIMO greatly improves the performance of several singlemodal and multi-modal downstream tasks. Our code and pre-trained models are public at https:

In this paper, the concept of sparse difference resultant for a Laurent transformally essential system of difference polynomials is introduced and a simple criterion for the existence of sparse difference resultant is given. The concept of transformally homogenous polynomial is introduced and the sparse difference resultant is shown to be transformally homogenous. It is shown that the vanishing of the sparse difference resultant gives a necessary condition for the corresponding difference polynomial system to have non-zero solutions. The order and degree bounds for sparse difference resultant are given. Based on these bounds, an algorithm to compute the sparse difference resultant is proposed, which is single exponential in terms of the number of variables, the Jacobi number, and the size of the Laurent transformally essential system. Furthermore, the precise order and degree, a determinant representation, and a Poisson-type product formula for the difference resultant are given.

Learned avoidance behavior is a critical component of animal survival; thus, any model of animal learning must account for the phenomenon. The two dominant theories of animal behavior during the early 20th century were Pavlovian conditioning (Pavlov, 1927) and Thorndikian law of effect (Thorndike, 1898). Pavlov proposed that all animal learning could be explained by conditioning (also referred to as stimulus substitution). Thorndike's law of effect was founded on response substitution. Each theory explained an important characteristic of behavior: anticipation in the case of Pavlovian conditioning or goal-directed behavior in the case of Thorndikian law of effect. However, neither theory could account for the entirety of animal behavior. Miller and Konorski (1928/1969), and eventually others, proposed that an explanation of animal behavior required not one theory or the other, but both, a two-process theory of learning. Many view Mowrer's (1947) two-factor theory of learning as the definitive two-process theory. According to this theory, the first process involves fear being classically conditioned to the conditioned stimulus (CS) associated with the punishment signal. That CS is called the warning signal because it reliably precedes the punishment signal and is learned to be a cue for the punishment signal. The animal's conditioned response (CR) to the warning signal, fear,