A Permutation-Equivariant Neural Network Architecture For Auction Design

Rahme, Jad; Jelassi, Samy; Bruna, Joan; Weinberg, S. Matthew

doi:10.1609/aaai.v35i6.16711

Cited by 12 publications

(3 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A well-known example is the success of convolutional neural networks (CNNs) on image problems due to their (approximate) translation invariance [19]. Many types of symmetries have been explored in the design of neural networks, such as permutation equivariance and invariance [15,36,37,39,57], rotational equivariance and invariance [13,17,45,46], and more [23,40,42,52]. Some works deal with multiple symmetries.…”

Section: Background and Related Work 21 Neural Network And Symmetriesmentioning

confidence: 99%

Why Self-Attention Is Natural for Sequence-to-Sequence Problems? A Perspective from Symmetries

Chao Ma,

Lexing Ying

2023

JML

View full text Add to dashboard Cite

In this paper, we show that structures similar to self-attention are natural for learning many sequence-to-sequence problems from the perspective of symmetry. Inspired by language processing applications, we study the orthogonal equivariance of seq2seq functions with knowledge, which are functions taking two inputs -an input sequence and a knowledge -and outputting another sequence. The knowledge consists of a set of vectors in the same embedding space as the input sequence, containing the information of the language used to process the input sequence. We show that orthogonal equivariance in the embedding space is natural for seq2seq functions with knowledge, and under such equivariance, the function must take a form close to self-attention. This shows that network structures similar to self-attention are the right structures for representing the target function of many seq2seq problems. The representation can be further refined if a finite information principle is considered, or a permutation equivariance holds for the elements of the input sequence.

show abstract

Section: Background and Related Work 21 Neural Network And Symmetriesmentioning

confidence: 99%

Why Self-Attention Is Natural for Sequence-to-Sequence Problems? A Perspective from Symmetries

Chao Ma,

Lexing Ying

2023

JML

View full text Add to dashboard Cite

show abstract

“…As the space of auctions M may be large, Implementation details can be found in ; Rahme et al (2021b) or in Sec 5.4.…”

Section: Auction Design As a Learning Problemmentioning

confidence: 99%

Learning Algorithms for Intelligent Agents and Mechanisms

Rahme¹

2022

Preprint

View full text Add to dashboard Cite

The ability to learn from past experiences and adapt one's behavior accordingly within an environment or context to achieve a certain goal is a characteristic of a truly intelligent entity. Developing efficient, robust, and reliable learning algorithms towards that end is an active area of research and a major step towards achieving artificial general intelligence. In this thesis, we research learning algorithms for optimal decision making in two different contexts, Reinforcement Learning in Part I and Auction Design in Part II.Reinforcement learning (RL) is an area of machine learning that is concerned with how an agent should act in an environment in order to maximize its cumulative reward over time. In Chapter 2, inspired by statistical physics, we develop a novel approach to RL that not only learns optimal policies with enhanced desirable properties but also sheds new light on maximum entropy RL. In Chapter 3, we tackle the generalization problem in RL using a Bayesian perspective. We show that imperfect knowledge of the environment's dynamics effectively turn a fully-observed Markov Decision Process (MDP) into a Partially Observed MDP (POMDP) that we call the Epistemic POMDP.Informed by this observation, we develop a new policy learning algorithm LEEP which has improved generalization properties.An auction is the process of organizing the buying and selling of products and services that is of great practical importance. Designing an incentive compatible, individually rational auction that maximizes revenue is a challenging and intractable problem. Recently, a deep learning based approach was proposed to learn optimal auctions from data. While successful, this approach suffers from a few limitations, including sample inefficiency, lack of generalization to new auctions, and training difficulties. In Chapter 4, we construct a symmetry preserving neural network architecture, EquivariantNet, suitable for anonymous auctions. EquivariantNet is not only more sample efficient but is also able to learn auction rules that generalize well to iii other settings. In Chapter 5, we propose a novel formulation of the auction learning problem as a two player game. The resulting learning algorithm, ALGNet, is easier to train, more reliable and better suited for non stationary settings. iv First of all, I would like to thank my adviser, Ryan Adams, for his guidance and support throughout my doctoral studies at Princeton. I'm grateful for his availability, flexibility, and generosity, especially when it comes to sharing research ideas and insights on a wide range of topics.Throughout my PhD, I was fortunate to have the support of many Princeton faculty and administrative staff. I'm grateful to Matt Weinberg for his guidance and mentorship. His knowledge and expertise in mechanism design were critical when it came to bringing the second part of this thesis to fruition. I would also like to thank Peter Ramadge, Szymon Rusinkiewicz, Karthik Narasimhan for completing my thesis committee. I extend my gratitude to the supportive PACM...

show abstract

“…More recently, leaning on the concept of differentiable programming as applied to economics, Düetting et al, 6 and follow-on work, uses modern deep learning to design differentiable-economics-based approaches to automated mechanism design, wherein a deep network is used as an approximator to a learned auction, voting rule, matching policy, or otherwise. While promising, there are many open questions [7][8][9][10][11] in this space having to do with the manipulation of learned mechanisms, fairness considerations, robustness to noise, scalability, and others.…”

Section: Introductionmentioning

confidence: 99%

Trustworthy agents under a veil of misinformation: mechanism design under adversarial upstream contitions

Richardson

Mittrick

Dickerson

et al. 2023

Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications V

View full text Add to dashboard Cite

In many situations, agents are on the same team as the central decision-maker; they express preferences truthfully to that decision-maker, who then aggregates those expressed preferences to decide on an outcome. This strays from, and is easier to analyze than, the traditional mechanism design setting from microeconomics, where agents are not assumed to be truthful. But, what if those trusted agents' true preferences are manipulated by upstream actors (either intentional or not)? How should a decision-maker act when either strategic or natural uncertainty manipulates trusted agents' beliefs? In this brief position paper, we propose a new area of focus for mechanism designers that captures a variety of real-world settings where agents are "on the same team" but can be manipulated by an exogenous actor.

show abstract

A Permutation-Equivariant Neural Network Architecture For Auction Design

Cited by 12 publications

References 53 publications

Why Self-Attention Is Natural for Sequence-to-Sequence Problems? A Perspective from Symmetries

Why Self-Attention Is Natural for Sequence-to-Sequence Problems? A Perspective from Symmetries

Learning Algorithms for Intelligent Agents and Mechanisms

Trustworthy agents under a veil of misinformation: mechanism design under adversarial upstream contitions

Contact Info

Product

Resources

About