Submodularity in Action: From Machine Learning to Signal Processing Applications

Abstract

ubmodularity is a discrete domain functional property that can be interpreted as mimicking the role of well-known convexity/concavity properties in the continuous domain. Submodular functions exhibit strong structure that lead to efficient optimization algorithms with provable near-optimality guarantees. These characteristics, namely, efficiency and provable performance bounds, are of particular interest for signal processing (SP) and machine learning (ML) practitioners, as a variety of discrete optimization problems are encountered in a wide range of applications. Conventionally, two general approaches exist to solve discrete problems: 1) relaxation into the continuous domain to obtain an approximate solution or 2) the development of a tailored algorithm that applies directly in the discrete domain. In both approaches, worst-case performance guarantees are often hard to establish. Furthermore, they are often complex and thus not practical for large-scale problems. In this article, we show how certain scenarios lend themselves to exploiting submodularity for constructing scalable solutions with provable worst-case performance guarantees. We introduce a variety of submodular-friendly applications and elucidate the relation of submodularity to convexity and concavity, which enables efficient optimization. With a mixture of theory and practice, we present different flavors of submodularity accompanying illustrative real-world case studies from modern SP and ML. In all of the cases, optimization algorithms are presented along with hints on how optimality guarantees can be established.

PLATINUM: Semi-Supervised Model Agnostic Meta-Learning using Submodular Mutual Information

“…Submodular functions. Submodular functions (Tohidi et al, 2020;Bach, 2011;2019) have been widely used for data subset selection as they naturally model properties like coverage, representation, diversity, etc.. Given a ground-set of n data points…”

Sampling Graph Signals with Sparse Dictionary Representation

“…As a result, we can directly apply the greedy heuristic to problem (P2) by adding to the sampling set the residual node n ∈ Si = V \ S i that minimizes the MSD. Alternatively, other criteria used in experimental design that exhibit amenable properties for greedy selection (e.g., submodularity [20]), such as the (pseudo) log-determinant criterion…”

“…This is because the support estimated from (P Si,0 ) may change between iterations; especially in the earlier ones. However, when the number of samples becomes large enough and the support does not change, using submodular functions in (P2) may come with near-optimal guarantees [20]. A deeper analysis of the latter will be done in future work.…”

Long Short-Term Memory Neural Equalizer

“…Model based equalizers usually requires a good understanding of the transmission systems attributions and a rich expert knowledge in design but often lack of generalization ability [9]. Facing this challenge with the emergence of artificial intelligence technology [10], several learning-based equalizers have been proposed that are more adaptable over varied channel conditions. Most of the learning-based equalizers are nonlinear equalizers.…”