Greedy maximal independent sets via local limits

Krivelevich, Michael; Mészáros, Tamás; Michaeli, Peleg; Shikhelman, Clara

doi:10.1002/rsa.21200

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article1

Preprint1

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Evaluating generalizability of artificial intelligence models for molecular datasets

Ektefaie,

Shen,

Bykova

et al. 2024

Nat Mach Intell

View full text Add to dashboard Cite

Evaluating generalizability of artificial intelligence models for molecular datasets

Ektefaie,

Shen,

Bykova

et al. 2024

Nat Mach Intell

View full text Add to dashboard Cite

Evaluating generalizability of artificial intelligence models for molecular datasets

Ektefaie,

Shen,

Bykova

et al. 2024

Preprint

View full text Add to dashboard Cite

Deep learning has made rapid advances in modeling molecular sequencing data. Despite achieving high performance on benchmarks, it remains unclear to what extent deep learning models learn general principles and generalize to previously unseen sequences. Benchmarks traditionally interrogate model generalizability by generating metadata based (MB) or sequence-similarity based (SB) train and test splits of input data before assessing model performance. Here, we show that this approach mischaracterizes model generalizability by failing to consider the full spectrum of cross-split overlap,i.e., similarity between train and test splits. We introduce SPECTRA, a spectral framework for comprehensive model evaluation. For a given model and input data, SPECTRA plots model performance as a function of decreasing cross-split overlap and reports the area under this curve as a measure of generalizability. We apply SPECTRA to 18 sequencing datasets with associated phenotypes ranging from antibiotic resistance in tuberculosis to protein-ligand binding to evaluate the generalizability of 19 state-of-the-art deep learning models, including large language models, graph neural networks, diffusion models, and convolutional neural networks. We show that SB and MB splits provide an incomplete assessment of model generalizability. With SPECTRA, we find as cross-split overlap decreases, deep learning models consistently exhibit a reduction in performance in a task- and model-dependent manner. Although no model consistently achieved the highest performance across all tasks, we show that deep learning models can generalize to previously unseen sequences on specific tasks. SPECTRA paves the way toward a better understanding of how foundation models generalize in biology.

show abstract

Greedy maximal independent sets via local limits

Cited by 2 publications

References 68 publications

Evaluating generalizability of artificial intelligence models for molecular datasets

Evaluating generalizability of artificial intelligence models for molecular datasets

Evaluating generalizability of artificial intelligence models for molecular datasets

Contact Info

Product

Resources

About