Generating synthetic data in finance

Assefa, Samuel; Dervovic, Danial; Mahfouz, Mahmoud; Tillman, Robert E.; Reddy, Prashant; Veloso, Manuela

doi:10.1145/3383455.3422554

Cited by 99 publications

(41 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Transaction. This dataset is collected from the anti-money laundering (AML) financial system [3,6] which provides transaction records between users over time. At each timestamp, we construct a transaction network to represent the transactions occurring inside this timestamp.…”

Section: A Analysismentioning

confidence: 99%

A Look into Causal Effects under Entangled Treatment in Graphs: Investigating the Impact of Contact on MRSA Infection

Chen

Vullikanti

et al. 2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Methicillin-resistant Staphylococcus aureus (MRSA) is a type of bacteria resistant to certain antibiotics, making it difficult to prevent MRSA infections. Among decades of efforts to conquer infectious diseases caused by MRSA, many studies have been proposed to estimate the causal effects of close contact (treatment) on MRSA infection (outcome) from observational data. In this problem, the treatment assignment mechanism plays a key role as it determines the patterns of missing counterfactuals -the fundamental challenge of causal effect estimation. Most existing observational studies for causal effect learning assume that the treatment is assigned individually for each unit. However, on many occasions, the treatments are pairwisely assigned for units that are connected in graphs, i.e., the treatments of different units are entangled. Neglecting the entangled treatments can impede the causal effect estimation. In this paper, we study the problem of causal effect estimation with treatment entangled in a graph. Despite a few explorations for entangled treatments, this problem still remains challenging due to the following challenges: (1) the entanglement brings difficulties in modeling and leveraging the unknown treatment assignment mechanism; (2) there may exist hidden confounders which lead to confounding biases in causal effect estimation; (3) the observational data is often time-varying. To tackle these challenges, we propose a novel method NEAT, which explicitly leverages the graph structure to model the treatment assignment mechanism, and mitigates confounding biases based on the treatment assignment modeling. We also extend our method into a dynamic setting to handle timevarying observational data. Experiments on both synthetic datasets and a real-world MRSA dataset validate the effectiveness of the proposed method, and provide insights for future applications. CCS CONCEPTS• Applied computing → Law, social and behavioral sciences;• Mathematics of computing → Causal networks.

show abstract

Section: A Analysismentioning

confidence: 99%

A Look into Causal Effects under Entangled Treatment in Graphs: Investigating the Impact of Contact on MRSA Infection

Chen

Vullikanti

et al. 2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

show abstract

“…The highly-sensitive nature of the financial data and the resulting access restrictions are major obstacles in this process. Latest proposals attempt to tackle these issues by improving the quality of synthetic data [9] and building legal-technical frameworks in order to balance the opposing openness and privacy forces [135]. (iii) Non-Profits and AI Consortia: Lately, a number of non-profits and AI consortia has been formed to primarily focus on responsible and trustworthy AI development (e.g.…”

Section: Ai Ethics Organizationsmentioning

confidence: 99%

On the Current and Emerging Challenges of Developing Fair and Ethical AI Solutions in Financial Services

Kurshan,

Chen,

Storchan

et al. 2021

Preprint

View full text Add to dashboard Cite

AI has found a wide range of application areas in the financial services industry. As the number and the criticality of the applications continue to increase, fair and ethical AI has emerged as an industry-wide objective. In recent years, numerous ethical principles, guidelines and techniques have been proposed. However, the model development organizations face serious challenges in building ethical AI solutions. This paper focuses on the overarching issues model development teams face, which range from the design and implementation complexities, to the shortage of tools, and the lack of organizational constructs. It argues that focusing on the practical considerations is an important step in bridging the gap between the high-level ethics principles and the deployed AI applications, as well as starting industry-wide conversations toward solution approaches.

show abstract

“…In this context, the term 'synthetic data' denotes data of a generative process grounded in the inherent properties of real data. Modeling these processes provides a nuanced understanding of underlying patterns, unlocking insights, especially in high-stake domains (Assefa et al 2020;Schreyer et al 2019). This idea stands apart from conventional data obfuscation methods such as anonymization or eliminating sensitive attributes.…”

Section: Introductionmentioning

confidence: 99%

“…To effectively train these models, characterized by extensive parameters, substantial compute resources, and extensive training data are crucial (de Goede 2023). However, challenges arise when sensitive data is distributed across various institutions, such as hospitals, municipalities, and financial authorities, and cannot be shared due to privacy concerns (Assefa et al 2020;Schreyer, Sattarov, and Borth 2022). Recently, the concept of Federated Learning (FL) was proposed in which multiple devices, such as smartphones or servers, collaboratively train AI models under the orchestration of a central server (Kairouz et al 2019).…”

Section: Introductionmentioning

confidence: 99%

FinDiff: Diffusion Models for Financial Tabular Data Generation

Sattarov,

Schreyer,

Borth

2023

4th ACM International Conference on AI in Finance

View full text Add to dashboard Cite

Realistic synthetic tabular data generation encounters significant challenges in preserving privacy, especially when dealing with sensitive information in domains like finance and healthcare. In this paper, we introduce Federated Tabular Diffusion (FedTabDiff) for generating high-fidelity mixed-type tabular data without centralized access to the original tabular datasets. Leveraging the strengths of Denoising Diffusion Probabilistic Models (DDPMs), our approach addresses the inherent complexities in tabular data, such as mixed attribute types and implicit relationships. More critically, FedTabDiff realizes a decentralized learning scheme that permits multiple entities to collaboratively train a generative model while respecting data privacy and locality. We extend DDPMs into the federated setting for tabular data generation, which includes a synchronous update scheme and weighted averaging for effective model aggregation. Experimental evaluations on realworld financial and medical datasets attest to the framework's capability to produce synthetic data that maintains high fidelity, utility, privacy, and coverage.

show abstract

Generating synthetic data in finance

Cited by 99 publications

References 37 publications

A Look into Causal Effects under Entangled Treatment in Graphs: Investigating the Impact of Contact on MRSA Infection

A Look into Causal Effects under Entangled Treatment in Graphs: Investigating the Impact of Contact on MRSA Infection

On the Current and Emerging Challenges of Developing Fair and Ethical AI Solutions in Financial Services

FinDiff: Diffusion Models for Financial Tabular Data Generation

Contact Info

Product

Resources

About