Training strategy for unbalanced small datasets in deep learning

Su, Bo‐Han; Liu, Hsin; Lee, Cheng-Wei

doi:10.1021/scimeetings.0c07068

Search citation statements

Order By: Relevance

Paper Sections

Select...

Discussion1

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because each module makes equal but small contributions, a few neurons in a single layer are unlikely to explain how a deep learning model makes a particular prediction. It is therefore crucial to take all layers collectively for interpretation ( 37 ). This view, however, challenges the widely used layer-wise approaches to deep learning interpretation ( 38 , 39 ).…”

Section: Discussionmentioning

confidence: 99%

A law of data separation in deep learning

He,

2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

While deep learning has enabled significant advances in many areas of science, its black-box nature hinders architecture design for future artificial intelligence applications and interpretation for high-stakes decision-makings. We addressed this issue by studying the fundamental question of how deep neural networks process data in the intermediate layers. Our finding is a simple and quantitative law that governs how deep neural networks separate data according to class membership throughout all layers for classification. This law shows that each layer improves data separation at a constant geometric rate, and its emergence is observed in a collection of network architectures and datasets during training. This law offers practical guidelines for designing architectures, improving model robustness and out-of-sample performance, as well as interpreting the predictions.

show abstract

Section: Discussionmentioning

confidence: 99%

A law of data separation in deep learning

He,

2023

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

Training strategy for unbalanced small datasets in deep learning

Cited by 1 publication

References 0 publications

A law of data separation in deep learning

A law of data separation in deep learning

Contact Info

Product

Resources

About