Skillearn: Machine Learning Inspired by Humans' Learning Skills

Abstract

<div>Humans, as the most powerful learners on the planet, have accumulated a lot of learning skills, such as learning through tests, interleaving learning, self-explanation, active recalling, to name a few. These learning skills and methodologies enable humans to learn new topics more effectively and efficiently. We are interested in investigating whether humans' learning skills can be borrowed to help machines to learn better. Specifically, we aim to formalize these skills and leverage them to train better machine learning (ML) models. To achieve this goal, we develop a general framework -- Skillearn, which provides a principled way to represent humans' learning skills mathematically and use the formally-represented skills to improve the training of ML models. In two case studies, we apply Skillearn to formalize two learning skills of humans: learning by passing tests and interleaving learning, and use the formalized skills to improve neural architecture search. Experiments on various datasets show that trained using the skills formalized by Skillearn, ML models achieve significantly better performance.</div>

Learning by Examples Based on Multi-level Optimization

“…BLO has been extended to multi-level optimization (MLO) which involves more than two levels of optimization problems. MLO has been applied for data generation [58], interleaving multi-task learning [2], data reweighting in domain adaptation [68], explainable learning [23], humaninspired learning [66], curriculum evaluation [13], mutual knowledge distillation [12], end-to-end knowledge distillation [55], etc.…”