A Survey on Curriculum Learning

Wang, Xin; Chen, Yudong; Zhu, Wenwu

doi:10.1109/tpami.2021.3069908

Cited by 281 publications

(131 citation statements)

References 126 publications

(197 reference statements)

Supporting

Mentioning

128

Contrasting

Order By: Relevance

“…Meanwhile, several studies have proposed similar concerns. Wang et al (2021b) raised a similar question about "easy-first versus hard-first" under the context of curriculum learning. This paper explores this question from a global perspective, obtaining reasonable findings.…”

Section: Existing Weighting Schemesmentioning

confidence: 99%

Which Samples Should be Learned First: Easy or Hard?

Zhou¹,

Wu²

2021

Preprint

View full text Add to dashboard Cite

An effective weighting scheme for training samples is essential for learning tasks. Numerous weighting schemes have been proposed. Some schemes take the easyfirst mode on samples, whereas some others take the hard-first mode. Naturally, an interesting yet realistic question is raised. Which samples should be learned first given a new learning task, easy or hard? To answer this question, three aspects of research are carried out. First, a high-level unified weighted loss is proposed, providing a more comprehensive view for existing schemes. Theoretical analysis is subsequently conducted and preliminary conclusions are obtained. Second, a flexible weighting scheme is proposed to overcome the defects of existing schemes. The three modes, namely, easy/medium/hard-first, can be flexibly switched in the proposed scheme. Third, a wide range of experiments are conducted to further compare the weighting schemes in different modes. On the basis of these works, reasonable answers are obtained. Factors including prior knowledge and data characteristics determine which samples should be learned first in a learning task.

show abstract

Section: Existing Weighting Schemesmentioning

confidence: 99%

Which Samples Should be Learned First: Easy or Hard?

Zhou¹,

Wu²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Curriculum learning (CL) [2,40] is a training strategy that trains models initially on easy data and then on harder data, imitating how human students are taught with curricula. It could help gradientbased models to escape local minimum to some extent in lowervariance gradient directions [42].…”

Section: Curriculum Learning and Hard Sample Miningmentioning

confidence: 99%

“…In each stage, we first update the meta-learner in a MAML-like manner, and then conditionally re-sample a new batch of tasks containing the most difficult users (Stage 1) and cities (Stage 2) under a loss-based criterion. Secondly, we further propose to draw upon the curriculum learning [40] strategies to help improve the convergence rate and generalization capacity of the meta-learner under the condition of high city-level diversity, which boosts the transfer performance when the amount of base cities is relatively large (and the diversity of these cities is thus relatively high). The basic idea is to present training tasks for the meta-learner in an easier first, harder later paradigm with the difficulties judged by a teacher, in order to help the meta-learner converge to a better state.…”

Section: Introductionmentioning

confidence: 99%

Curriculum Meta-Learning for Next POI Recommendation

Chen

Wang

Fan

et al. 2021

Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery &Amp; Data Mining

Self Cite

View full text Add to dashboard Cite

Next point-of-interest (POI) recommendation is a hot research field where a recent emerging scenario, next POI to search recommendation, has been deployed in many online map services such as Baidu Maps. One of the key issues in this scenario is providing satisfactory recommendation services for cold-start cities with a limited number of user-POI interactions, which requires transferring the knowledge hidden in rich data from many other cities to these cold-start cities. Existing literature either does not consider the city-transfer issue or cannot simultaneously tackle the data sparsity and pattern diversity issues among various users in multiple cities. To address these issues, we explore city-transfer next POI to search recommendation that transfers the knowledge from multiple cities with rich data to cold-start cities with scarce data. We propose a novel Curriculum Hardness Aware Meta-Learning (CHAML) framework, which incorporates hard sample mining and curriculum learning into a meta-learning paradigm. Concretely, the CHAML framework considers both city-level and user-level hardness to enhance the conditional sampling during meta training, and uses an easy-to-hard curriculum for the city-sampling pool to help the meta-learner converge to a better state. Extensive experiments on two real-world map search datasets from Baidu Maps demonstrate the superiority of CHAML framework.

show abstract

“…it believes that the expense of labeling operations mainly hinges on the total of queries, hence the objective is to reduce the number of queries as possible [60]. In contrast, deep semisupervised learning can be regarded as a learning paradigm that combines the advantages of both supervised learning and unsupervised learning by allowing the deep learning model to be trained using both unlabeled and labeled samples, without any human interventions [61,62]. Figure 1.9 display the workflow of the semisupervised deep learning model.…”

Section: Deep Weakly Supervised Learningmentioning

confidence: 99%

Introduction Conceptualization of Security, Forensics, and Privacy of Internet of Things: An Artificial Intelligence Perspective

Abdel‐Basset

Moustafa

Hawash

et al. 2021

Deep Learning Techniques for IoT Security and Privacy

View full text Add to dashboard Cite

The Internet of Things (IoT) is a rapidly evolving technology that empowers billions of globally distributed physical things to be interconnected over the internet to capture, collect, exchange, and share a wide variety of vast amounts of data. These physical things incorporate all of the connectable devices ranging from conventional household devices to complex industrial devices [1]. The emergence of INDUSTRY 4.0 has revolutionized the industrial domain globally with a large and ever-increasing number of embedded devices, and computerized chips because of the expanding availability, affordability, microprocessors, sensors' capacity, and omnipresent communication technologies. Connecting such a very large number of heterogeneous objects and implanting sensors to them leads to some degree of digital intelligence at devices that, empowering them to communicate instantaneous data with no human intervention. The IoT empowering the infrastructure of the globe to be more intelligent and reactive, combining the physical and digital domains. Therefore, the IoT has been receiving great interest and wide adoption in almost all fields: education, manufacturing, healthcare, business, entertainment, energy distribution, domestic, smart-cities, transportations, tourism, and even research [2].Regrettably, industry, academia, and people are struggling to integrate the stream of rapid commercialization with rare consideration to the security and the privacy of IoT data and devices. For example, think about an IoT-enabled world that combines things like wearables, smartphones, smartwatches, smart TVs, smart refrigerators, smart vehicles, implantable health devices, industrial robotics systems, and simply everything capable to be connected over networks. Generally, most of these industries have never posed any interest in security, forensics, or privacy concerns over the last decades [3]. Given the intense competition to be more feasible along with profitable products and services, nonetheless, the current industries discover themselves in the critical phase as they do not realize the secure and privacy-preserving method Electronic Supplementary Material The online version of this chapter

show abstract

A Survey on Curriculum Learning

Cited by 281 publications

References 126 publications

Which Samples Should be Learned First: Easy or Hard?

Which Samples Should be Learned First: Easy or Hard?

Curriculum Meta-Learning for Next POI Recommendation

Introduction Conceptualization of Security, Forensics, and Privacy of Internet of Things: An Artificial Intelligence Perspective

Contact Info

Product

Resources

About