Low-time-complexity document clustering using memristive dot product engine

Zhou, Houji; Li, Yang; Miao, Xiangshui

doi:10.1007/s11432-021-3316-x

Cited by 9 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, the prototype can be fine‐tuned in the memristor like the data clustering method in ref. [33] Our previous studies have also discussed the online training methods for prototypal clustering, [ 34,35 ] which introduce the basic ideas and detailed steps for online learning in the memristor array. Here, we prefer to use digital systems to update the prototypes to reduce complex peripheral circuits.…”

Section: Resultsmentioning

confidence: 99%

Memristive Cosine‐Similarity‐Based Few‐Shot Learning with Lifelong Memory Adaptation

Zhou

Ling

Bao

et al. 2023

Advanced Intelligent Systems

Self Cite

View full text Add to dashboard Cite

Deep learning in recent years has drawn much attention for its outperformance in the fields of computer vision, [1] speech recognition, [2] and natural language processing. [3] The implementation of these tasks relies heavily on the use of deep neural networks (DNNs) to model the prior knowledge distribution in massive amounts of data. In most cases, acquiring training data for DNNs is much more important than the DNN models. [4] Nevertheless, processing data with a limited size of training materials is more in line with the human brain, such as a kid can easily distinguish a dog from cats once he is taught what a dog looks like with one sample. But this problem faces great challenges for the mainstream neural networks because DNNs fail to learn the features of a dog in only one picture. Few-shot learning (FSL), a branch of meta-learning, [5] has been accordingly proposed to build reliable machine learning models using a few labeled examples, just like the way humans work. Among the popular FSL algorithms, the use of dynamic external memory combined with neural network control, [6] also known as memory-augmented neural network (MANN), [7] stands out for the outperformance on FSL tasks. The external memory, usually DRAM, [7,8] is used to remember the features of the insufficient samples for a long time and assists in improving the performance of FSL consistently. However, the time delay caused by the data communication between the memory and the running neural network is costly and degrades the performance of data search using external memory. The introduction of emerging non-volatile memory has become a viable solution to this problem due to its non-volatile nature, which greatly improves search speed and shows the potential for lifelong learning tasks.Various non-volatile devices, such as the flash, [9] FeFETs, [10,11] resistive random-access memory (RRAM), [12][13][14] and phase-change memory (PCM), [15] have been used as external memory to accelerate the similarity computation in MANN. Ni et al. [11] proposed a two-FeFET scheme as a ternary content addressable memory (ternary CAM) cell to calculate the Hamming distances of binary vectors for data indexing. However, two problems arising from this framework are the reduced accuracy compared to digital computers and the long encoding bits for the binarized vectors. Then Karunaratne et al. [15] designed a new attention mechanism to improve the accuracy of similarity-based FSL with the cosine distance of binarized vectors on PCMs. The length of the embedding vectors in refs. [11,15] are 512 bits with the Omniglot dataset, leading to a large area and power overhead of external memory. Kazemi et al. [10] developed an analog CAM with FeFET using the structure in ref. [11] and tried to reduce the encoding

show abstract

Section: Resultsmentioning

confidence: 99%

Memristive Cosine‐Similarity‐Based Few‐Shot Learning with Lifelong Memory Adaptation

Zhou

Ling

Bao

et al. 2023

Advanced Intelligent Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our IMS has an extremely low computation complexity because it needs only some read operations during the computing process instead of the complex floating‐point multiplications and additions as shown in Figure 5I. Although some previous studies have proposed efficient methods to calculate the Euclidean distance and Cosine distance using analogue memristors, yet still requires a complex write‐verify algorithm to map the values on the devices 30,31 . Besides, another benefit of binary computing is that it is better compatible with current technology both in hardware and operation.…”

Section: Discussionmentioning

confidence: 99%

“…Although some previous studies have proposed efficient methods to calculate the Euclidean distance and Cosine distance using analogue memristors, yet still requires a complex write-verify algorithm to map the values on the devices. 30,31 Besides, another benefit of binary computing is that it is better compatible with current technology both in hardware and operation. This can avoid using a large number of digital-analogue/analogue-digital converts in circuits.…”

Section: Discussionmentioning

confidence: 99%

Self‐selective memristor‐enabled in‐memory search for highly efficient data mining

Ling

Huang

et al. 2023

InfoMat

Self Cite

View full text Add to dashboard Cite

Similarity search, that is, finding similar items in massive data, is a fundamental computing problem in many fields such as data mining and information retrieval. However, for large‐scale and high‐dimension data, it suffers from high computational complexity, requiring tremendous computation resources. Here, based on the low‐power self‐selective memristors, for the first time, we propose an in‐memory search (IMS) system with two innovative designs. First, by exploiting the natural distribution law of the devices resistance, a hardware locality sensitive hashing encoder has been designed to transform the real‐valued vectors into more efficient binary codes. Second, a compact memristive ternary content addressable memory is developed to calculate the Hamming distances between the binary codes in parallel. Our IMS system demonstrated a 168× energy efficiency improvement over all‐transistors counterparts in clustering and classification tasks, while achieving a software‐comparable accuracy, thus providing a low‐complexity and low‐power solution for in‐memory data mining applications.

show abstract

“…The main challenge in accelerating the cosine similarities in the crossbar array lies in the L2-normalization part. Zhou et al first developed a cosine similarity calculation method based on the spherical data utilizing the RRAM arrays, and proposed an online training method for cosine similaritybased K-means data clustering [83]. The simulation results show its reliable capabilities in text classification and further data analysis for sparse vectors.…”

Section: Non-ann Based Memristive Machine Learning Algorithmsmentioning

confidence: 99%

Toward memristive in-memory computing: principles and applications

Bao

Zhou

et al. 2022

Front. Optoelectron.

Self Cite

View full text Add to dashboard Cite

With the rapid growth of computer science and big data, the traditional von Neumann architecture suffers the aggravating data communication costs due to the separated structure of the processing units and memories. Memristive in-memory computing paradigm is considered as a prominent candidate to address these issues, and plentiful applications have been demonstrated and verified. These applications can be broadly categorized into two major types: soft computing that can tolerant uncertain and imprecise results, and hard computing that emphasizes explicit and precise numerical results for each task, leading to different requirements on the computational accuracies and the corresponding hardware solutions. In this review, we conduct a thorough survey of the recent advances of memristive in-memory computing applications, both on the soft computing type that focuses on artificial neural networks and other machine learning algorithms, and the hard computing type that includes scientific computing and digital image processing. At the end of the review, we discuss the remaining challenges and future opportunities of memristive in-memory computing in the incoming Artificial Intelligence of Things era. Graphical Abstract

show abstract

Low-time-complexity document clustering using memristive dot product engine

Cited by 9 publications

References 35 publications

Memristive Cosine‐Similarity‐Based Few‐Shot Learning with Lifelong Memory Adaptation

Memristive Cosine‐Similarity‐Based Few‐Shot Learning with Lifelong Memory Adaptation

Self‐selective memristor‐enabled in‐memory search for highly efficient data mining

Toward memristive in-memory computing: principles and applications

Contact Info

Product

Resources

About