Accordion: A Trainable Simulator for Long-Term Interactive Systems

McInerney, James O.; Elahi, Ehtsham; Basilico, Justin; Raimond, Yves; Jebara, Tony

doi:10.1145/3460231.3474259

Cited by 14 publications

(10 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the usefulness of simulations, many simulation frameworks have been developed to study various fairness approaches for information retrieval systems; just to mention a few: MARS-Gym [139], ML-fairness-gym [41], Accordion [108], RecLab [90], RecSim NG [115], SIREN [23], T-RECS [105], RecoGym [137], AESim [59], Virtual-Taobao [146].…”

Section: Simulation and Applied Modeling To Study Long-term Effects A...mentioning

confidence: 99%

Fair ranking: a critical review, challenges, and future directions

Patro¹,

Porcaro²,

Mitchell³

et al. 2022

Preprint

View full text Add to dashboard Cite

Ranking, recommendation, and retrieval systems are widely used in online platforms and other societal systems, including e-commerce, media-streaming, admissions, gig platforms, and hiring. In the recent past, a large "fair ranking" research literature has been developed around making these systems fair to the individuals, providers, or content that are being ranked. Most of this literature defines fairness for a single instance of retrieval, or as a simple additive notion for multiple instances of retrievals over time. This work provides a critical overview of this literature, detailing the often context-specific concerns that such an approach misses: the gap between high ranking placements and true provider utility, spillovers and compounding effects over time, induced strategic incentives, and the effect of statistical uncertainty. We then provide a path forward for a more holistic and impact-oriented fair ranking research agenda, including methodological lessons from other fields and the role of the broader stakeholder community in overcoming data bottlenecks and designing effective regulatory environments.

show abstract

Section: Simulation and Applied Modeling To Study Long-term Effects A...mentioning

confidence: 99%

Fair ranking: a critical review, challenges, and future directions

Patro¹,

Porcaro²,

Mitchell³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Although when different system modules are changing rapidly and systems are getting more complex, it becomes more practical to simulate the effect than test it using traditional methods. [18] presents one of many ways to simulate interactive systems. The paper proposes Accordion, a fully trainable simulator for interactive systems based on inhomogeneous Poisson processes.…”

Section: Churn Predictionmentioning

confidence: 99%

Challenges and opportunities in applying Neural Temporal Point Processes to large scale industry data

Šeputis¹,

Gamper²,

Paulavičius³

2022

Preprint

View full text Add to dashboard Cite

In this work, we identify open research opportunities in applying Neural Temporal Point Process (NTPP) models to industry scale customer behavior data by carefully reproducing NTPP models published up to date on known literature benchmarks as well as applying NTPP models to a novel, real world consumer behavior dataset that is twice as large as the largest publicly available NTPP benchmark. We identify the following challenges. First, NTPP models, albeit their generative nature, remain vulnerable to dataset imbalances and cannot forecast rare events. Second, NTPP models based on stochastic differential equations, despite their theoretical appeal and leading performance on literature benchmarks, do not scale easily to large industry-scale data. The former is in light of previously made observations on deep generative models. Additionally, to combat a cold-start problem, we explore a novel addition to NTPP models -a parametrization based on static user features.

show abstract

“…We propose a set of industrial criteria based on the resent publication Bernardi et al (2021); McInerney et al (2021); Gauci et al (2018) and the requirements we developed based on our own industry experience in RSs for a financial institution with hundreds of thousands of clients.…”

Section: Simulator's Industry Effectivenessmentioning

confidence: 99%

“…A rather simple user-item response model, that is used for the generation of user feedback on the item that is recommended by an RS (the rating matrix is preliminary debiased), is proposed in SOFA Huang et al (2020). A more complex methodology for simulation of user visits, time of the visits and the number of interactions in each visit is proposed in Accordion McInerney et al (2021). This methodology is based on Poisson processes and is used for modeling user-item interaction sequences.…”

Section: Models For Generating User-item Responsesmentioning

confidence: 99%

“…Another effect that can be modeled is the increased (or decreased) amount of user visits. The amount of user visits essentially depends on the quality of the RS used McInerney et al (2021). In this direction, the papers Adomavicius et al (2021); Lee and Hosanagar (2019); Zhang et al (2020); Zhou et al (2021); Yao et al (2021) discuss how the M&S can be also used to study long-term effects of interactions between users and RSs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Synthetic Data-Based Simulators for Recommender Systems: A Survey

Stavinova¹,

Grigorievskiy²,

Volodkevich³

et al. 2022

Preprint

View full text Add to dashboard Cite

This survey aims at providing a comprehensive overview of the recent trends in the field of modeling and simulation (M&S) of interactions between users and recommender systems and applications of the M&S to the performance improvement of industrial recommender engines. We start with the motivation behind the development of frameworks implementing the simulations -simulators -and the usage of them for training and testing recommender systems of different types (including Reinforcement Learning ones). Furthermore, we provide a new consistent classification of existing simulators based on their functionality, approbation, and industrial effectiveness and moreover make a summary of the simulators found in the research literature. Besides other things, we discuss the building blocks of simulators: methods for synthetic data (user, item, user-item responses) generation, methods for what-if experimental analysis, methods and datasets used for simulation quality evaluation (including the methods that monitor and/or close possible simulation-to-reality gaps), and methods for summarization of experimental simulation results. Finally, this survey considers emerging topics and open problems in the field.

show abstract

Accordion: A Trainable Simulator for Long-Term Interactive Systems

Cited by 14 publications

References 10 publications

Fair ranking: a critical review, challenges, and future directions

Fair ranking: a critical review, challenges, and future directions

Challenges and opportunities in applying Neural Temporal Point Processes to large scale industry data

Synthetic Data-Based Simulators for Recommender Systems: A Survey

Contact Info

Product

Resources

About