Reinforcement Learning for Dynamic Bidding in Truckload Markets: an Application to Large-Scale Fleet Management with Advance Commitments

Wang, Yingfei; Nascimento, Juliana Martins Do; Powell, Warren

doi:10.48550/arxiv.1802.08976

Cited by 2 publications

(2 citation statements)

References 36 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Miller & Nie [8] present a solution that emphasizes the importance of integrating carrier competition, routing and bidding. Wang et al [28] design a reinforcement learning algorithm based on knowledge gradients to solve for a bidding structure with a broker intermediating between carriers and shippers. The broker aims to propose a price that satisfies both carrier and shipper, taking a percentage of accepted bids as its reward.…”

Section: Literature Reviewmentioning

confidence: 99%

Strategic bidding in freight transport using deep reinforcement learning

Heeswijk¹

2021

Preprint

View full text Add to dashboard Cite

This paper presents a multi-agent reinforcement learning algorithm to represent strategic bidding behavior in freight transport markets. Using this algorithm, we investigate whether feasible market equilibriums arise without any central control or communication between agents. Studying behavior in such environments may serve as a stepping stone towards self-organizing logistics systems like the Physical Internet. We model an agent-based environment in which a shipper and a carrier actively learn bidding strategies using policy gradient methods, posing bid-and ask prices at the individual container level. Both agents aim to learn the best response given the expected behavior of the opposing agent. A neutral broker allocates jobs based on bid-ask spreads.Our game-theoretical analysis and numerical experiments focus on behavioral insights. To evaluate system performance, we measure adherence to Nash equilibria, fairness of reward division and utilization of transport capacity. We observe good performance both in predictable, deterministic settings (∼95% adherence to Nash equilibria) and highly stochastic environments (∼85% adherence). Risk-seeking behavior may increase an agent's reward share, as long as the strategies are not overly aggressive. The results suggest a potential for full automation and decentralization of freight transport markets.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Strategic bidding in freight transport using deep reinforcement learning

Heeswijk¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Miller and Nie [5] present a solution that emphasizes the importance of integrating carrier competition, routing and bidding. Wang et al [16] design a reinforcement learning algorithm based on knowledge gradients to solve for a bidding structure with a broker intermediating between carriers and shippers. The broker aims to propose a price that satisfies both carrier and shipper, taking a percentage of accepted bids as its reward.…”

Section: Related Workmentioning

confidence: 99%

Smart Containers with Bidding Capacity: A Policy Gradient Algorithm for Semi-cooperative Learning

Heeswijk

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Smart modular freight containers -as propagated in the Physical Internet paradigm -are equipped with sensors, data storage capability and intelligence that enable them to route themselves from origin to destination without manual intervention or central governance. In this self-organizing setting, containers may autonomously place bids on transport services in a spot market setting. However, for individual containers it might be difficult to learn good bidding policies due to their short lifespan. By sharing information and costs between one another, smart containers can jointly learn bidding policies, even though simultaneously competing for the same transport capacity. We replicate this behavior by learning stochastic bidding policies in a semi-cooperative multi-agent setting. To this end, we develop a reinforcement learning algorithm based on the policy gradient framework. Numerical experiments show that sharing solely bids and acceptance decisions leads to stable bidding policies. Real-time system information only marginally improves performance; individual job properties suffice to place appropriate bids. Furthermore, we find that carriers may have incentives not to share information with the smart containers. The experiments give rise to several directions for follow-up research, particularly addressing the interaction between smart containers and transport services in selforganizing logistics.

show abstract

Reinforcement Learning for Dynamic Bidding in Truckload Markets: an Application to Large-Scale Fleet Management with Advance Commitments

Cited by 2 publications

References 36 publications

Strategic bidding in freight transport using deep reinforcement learning

Strategic bidding in freight transport using deep reinforcement learning

Smart Containers with Bidding Capacity: A Policy Gradient Algorithm for Semi-cooperative Learning

Contact Info

Product

Resources

About