Learning dynamic prices in electronic retail markets with customer segmentation

Abstract

In this paper, we use reinforcement learning (RL) techniques to determine dynamic prices in an electronic monopolistic retail market. The market that we consider consists of two natural segments of customers, captives and shoppers. Captives are mature, loyal buyers whereas the shoppers are more price sensitive and are attracted by sales promotions and volume discounts. The seller is the learning agent in the system and uses RL to learn from the environment. Under (reasonable) assumptions about the arrival process of customers, inventory replenishment policy, and replenishment lead time distribution, the system becomes a Markov decision process thus enabling the use of a wide spectrum of learning algorithms. In this paper, we use the Q-learning algorithm for RL to arrive at optimal dynamic prices that optimize the seller's performance metric (either long term discounted profit or long run average profit per unit time). Our model and methodology can also be used to compute optimal reorder quantity and optimal reorder point for the inventory policy followed by the seller and to compute the optimal volume discounts to be offered to the shoppers. Notation:(q, r ) Inventory policy, q = reorder quantity and r = reorder point λ Rate of Poisson arrival process of customers fFraction of customers who are captives pPosted price per unit item N Maximum number of items requested by backlogged customers

Dynamic pricing and learning: Historical origins, current research, and new directions

“…Machine-learning techniques that have been applied to dynamic pricing problems include evolutionary algorithms (Ramezani et al, 2011), particle swarm optimization (Mullen et al, 2006), reinforcement learning and Q-learning (Kutschinski et al, 2003, Raju et al, 2006, Könönen, 2006, Chinthalapati et al, 2006, Schwind, 2007, Cheng, 2008, Han et al, Vengerov, 2008, Cheng, 2009, Jintian and Lei, 2009, Han, 2010, Collins and Thomas, 2012, simulated annealing (Xia and Dube, 2007), Markov chain Monte Carlo methods (Chung et al, 2012), the aggregating algorithm (Levina et al, 2009) by Vovk (1990), goal-directed and derivative-following strategies in simulation (DiMicco et al, 2003), neural networks (Brooks et al, 1999, Kong, 2004, Ghose and Tran, 2009, Liu and Wang, 2013, and direct search methods (Brooks et al, 1999, Brooks et al, 2002.…”

Optional Insurance Compensation Rate Selection and Evaluation in Financial Institutions

“…The data can be either historical data or exogenous data; exogenous data can be obtained by experimental method and research method [12][13]. The implementation process of customer segmentation is based on sample learning method.…”

Dynamic pricing models for electronic business

“…Raju, Narahari, and Ravikumar [65][66][67][68] look at electronic retail markets with a single seller (without competition). The seller has an inventory of products which he replenishes according to a standard inventory policy.…”

“…• A retail market where there are two natural segments of customers, captives and shoppers [67,68]. Captives are mature, loyal buyers whereas shoppers are more price-sensitive and are attracted by sales promotions and volume discounts.…”

“…Raju, Narahari, and Ravikumar [65,75] consider an electronic retail market, where multiple sellers compete on price to attract incoming buyers. In the markets considered, sellers employ automated pricing agents or pricebots.…”