Boosting binary masks for multi-domain learning through affine transformations

Abstract

In this work, we present a new, algorithm for multi-domain learning. Given a pretrained architecture and a set of visual domains received sequentially, the goal of multi-domain learning is to produce a single model performing a task in all the domains together. Recent works showed how we can address this problem by masking the internal weights of a given original conv-net through learned binary variables. In this work, we provide a general formulation of binary mask based models for multi-domain learning by affine transformations of the original network parameters. Our formulation obtains significantly higher levels of adaptation to new domains, achieving performances comparable to domain-specific models while requiring slightly more than 1 bit per network parameter per additional domain. Experiments on two popular benchmarks showcase the power of our approach, achieving performances close to state-of-the-art methods on the Visual Decathlon Challenge.

Budget-Aware Pruning for Multi-domain Learning

“…In test time, the learned binary mask is multiplied by the weights of the convolutional layer. Expanding on this idea, Mancini et al [7,8] also makes use of masks, however, they learn an affine transformation of the weights through the use of the mask and some extra parameters. Focusing on increasing the accuracy with masks, Chattopadhyay et al [2] proposes a soft-overlap loss to encourage the masks to be domain-specific by minimizing the overlap between them.…”

Cluster-driven Graph Federated Learning over Multiple Domains

“…Since our model can identify data clusters through the previously described procedure, we can design a way to specialize the function f θ to each domain. Inspired by multi-domain learning [36,39,37,27,29], we can achieve this with domain-specific components. For simplicity, let us consider the parameters θ to be split into two sets, i.e.…”

“…Note that θ s is actually a set θ s = {θ d s } D d=1 where θ d s are the parameters specific to the d-th domain. To tailor the model to a specific domain, we can consider multiple ways to include θ s , such as direct influence on the agnostic parameters θ a [39,27,29] or residual activations [36,37]. Here we follow the latter strategy, since the former relies on the robustness of θ a , which is harder to guarantee in FL.…”