Optimizing debt collections using constrained reinforcement learning

Abe, Naoki; Melville, Prem; Pendus, Cezar; Reddy, Chandan K.; Jensen, David L.; Thomas, V.; Bennett, James J.; Anderson, Gary F.; Cooley, Brent; Kowalczyk, Melissa; Domick, Mark; Gardinier, Timothy

doi:10.1145/1835804.1835817

Cited by 54 publications

(54 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, note that the segmentation in the initial iteration is optimized with respect to the estimation of immediate rewards, which is clearly suboptimal for estimating the long-term reward. This is also confirmed in empirical evaluation using multiple data sets, as Abe et al (2010) discuss.…”

Section: Solution Descriptionsupporting

confidence: 67%

“…We refer the reader to Abe et al (2010) for the technical details. As we mention above, we use the MDP framework (Puterman 1994, Tsitsiklis andVan Roy 1997) to formulate the process of tax collections.…”

Section: Solution Descriptionmentioning

confidence: 99%

“…Therefore, given the standard version of this algorithm, its constrained version can be derived by replacing the maximization by the appropriate constrained optimization, giving rise to a specific instantiation of the general idea described in Figure 3. Abe et al (2010) provide details of this method.…”

Section: Appendix the Core Algorithmmentioning

confidence: 99%

See 2 more Smart Citations

Tax Collections Optimization for New York State

et al. 2012

Self Cite

View full text Add to dashboard Cite

The New York State Department of Taxation and Finance (NYS DTF) collects over $1 billion annually in assessed delinquent taxes. The mission of DTF's Collections and Civil Enforcement Division (CCED) is to increase collections, but to do so in a manner that respects the rights of citizens, by taking actions commensurate with each debtor's situation. CCED must accomplish this in an environment with limited resources. In a collaborative work, NYS DTF, IBM Research, and IBM Global Business Services developed a novel tax collection optimization solution to address this challenge. The operations research-based solution combines data analytics and optimization using the unifying framework of constrained Markov decision processes (C-MDP). The system optimizes the collection actions of agents with respect to maximizing long-term returns, while taking into account the complex dependencies among business needs, resources, and legal constraints. It generates a customized collections policy instead of broad-brush rules, thereby improving both the efficiency and adaptiveness of the collections process. It also enhances and improves the tax agency's ability to administer taxes equitably across the broad scope of individual taxpayers' situations. The system became operational in December 2009; from 2009 to 2010, New York State increased its collections from delinquent revenue by $83 million (8 percent) using the same set of resources. Given a typical annual increase of 2 to 4 percent, the system's expected benefit is approximately $120 to $150 million over a period of three years, far exceeding the initial target of $90 million.

show abstract

Section: Solution Descriptionsupporting

confidence: 67%

Section: Solution Descriptionmentioning

confidence: 99%

See 1 more Smart Citation

Tax Collections Optimization for New York State

et al. 2012

Self Cite

View full text Add to dashboard Cite

show abstract

“…Along the same lines, Abe et al (2010) present a safe RL method that enforces high-level business and legal constraints during each value iteration step of the RL process, and apply their method to a tax collection optimization problem. ARL is complementary to the approaches proposed in (Abe et al, 2011;Castro et al, 2012;Moldovan and Abbeel, 2012) as it supports different types of constraints. In particular, the PCTLencoded constraints used by our approach are not specific to any domain or application like those from (Castro et al, 2012) and (Abe et al, 2011).…”

Section: Related Workmentioning

confidence: 99%

“…The existing results from this area focus on specifying bounds for the reward obtained by the RL agent or for simple measures associated with this reward (Abe et al, 2011;Castro et al, 2012;Delage and Mannor, 2010;Geibel, 2006;Moldovan and Abbeel, 2012;Ponda et al, 2013). In contrast to these approaches, ARL uses probabilistic model checking to formally establish safe AMDP policies associated with the broad range of safety, reliability and performance constraints that can be formally specified in PCTL (Hansson and Jonsson, 1994) extended with rewards (Andova et al, 2004).…”

Section: Introductionmentioning

confidence: 99%

Assured Reinforcement Learning with Formally Verified Abstract Policies

Călinescu¹,

Kudenko²,

Banks³

2017

Proceedings of the 9th International Conference on Agents and Artificial Intelligence

View full text Add to dashboard Cite

Markov Decision Processes. Abstract:We present a new reinforcement learning (RL) approach that enables an autonomous agent to solve decision making problems under constraints. Our assured reinforcement learning approach models the uncertain environment as a high-level, abstract Markov decision process (AMDP), and uses probabilistic model checking to establish AMDP policies that satisfy a set of constraints defined in probabilistic temporal logic. These formally verified abstract policies are then used to restrict the RL agent's exploration of the solution space so as to avoid constraint violations. We validate our RL approach by using it to develop autonomous agents for a flag-collection navigation task and an assisted-living planning problem.

show abstract