Automating Analogue AI Chip Design with Genetic Search

Krestinskaya, Olga; Salama, Khaled N.; James, Alex Pappachen

doi:10.1002/aisy.202000075

Cited by 12 publications

(6 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because the size of the crossbars and activation functions can compensate for hardware non-idealities. The selection of the most robust architecture can be performed using Genetic Search as proposed in [39], where circuit non-idealities are included to the initial selection of the neural network architecture before hardware design.…”

Section: Discussion: Methods To Avoid the Effect Of Non-idealitiesmentioning

confidence: 99%

Analogue neuro-memristive convolutional dropout nets

Krestinskaya

James

2020

Proc. R. Soc. A.

Self Cite

View full text Add to dashboard Cite

Randomly switching neurons ON/OFF while training and inference process is an interesting characteristic of biological neural networks, that potentially results in inherent adaptability and creativity expressed by human mind. Dropouts inspire from this random switching behaviour and in the artificial neural network they are used as a regularization techniques to reduce the impact of over-fitting during the training. The energy-efficient digital implementations of convolutional neural networks (CNN) have been on the rise for edge computing IoT applications. Pruning larger networks and optimization for performance accuracy has been the main direction of work in this field. As opposed to this approach, we propose to build a near-sensor analogue CNN with high-density memristor crossbar arrays. Since several active elements such as amplifiers are used in analogue designs, energy efficiency becomes a main challenge. To address this, we extend the idea of using dropouts in training to also the inference stage. The CNN implementations require a subsampling layer, which is implemented as a mean pooling layer in the design to ensure lower energy consumption. Along with the dropouts, we also investigate the effect of non-idealities of memristor and that of the network.

show abstract

Section: Discussion: Methods To Avoid the Effect Of Non-idealitiesmentioning

confidence: 99%

Analogue neuro-memristive convolutional dropout nets

Krestinskaya

James

2020

Proc. R. Soc. A.

Self Cite

View full text Add to dashboard Cite

show abstract

“…A more efficient approach was to use a generic search approach to automate the model parameter selection, which will be our goal in the future work. [ 39 ]…”

Section: Methodsmentioning

confidence: 99%

“…A software tool is needed to incorporate these parameters when engineering ReRAM bitcells and exploring analog design choices, which can help designers and engineers understand these complex interactions and help make some broad tradeoffs and aid chip design. [ 29,39 ] Table 1 shows a detailed list of design considerations discussed earlier.…”

Section: Simulator For Dnn Accelerationmentioning

confidence: 99%

A Fully Integrated System‐on‐Chip Design with Scalable Resistive Random‐Access Memory Tile Design for Analog in‐Memory Computing

Cai

Yen

Uppala

et al. 2022

Advanced Intelligent Systems

View full text Add to dashboard Cite

As the demands of big data applications and deep learning continue to rise, the industry is increasingly looking to artificial intelligence (AI) accelerators. Analog in‐memory computing (AiMC) with emerging nonvolatile devices enable good hardware solutions, due to its high energy efficiency in accelerating the multiply‐and‐accumulation (MAC) operation. Herein, an Applied Materials custom‐designed system‐on‐chip (SoC) targeting AI applications with analog in‐memory computing using resistive random‐access memory (ReRAM) as the compute element is demonstrated. The first silicon achieves high energy efficiency in MAC operations. This chip is implemented with LeNet‐1 neural network on ReRAM tiles and demonstrated by Modified National Institute of Standards and Technology (MNIST) classification with accuracy matching that predicted in the simulations. A simulation framework, AI Sim, is also developed to evaluate the system performance for large‐scale application and guide the bitcell development and design choices.

show abstract

“…These software-related design parameters include the number of neurons and layers in the network, the sizes of convolution kernels, activation functions, etc. For example, memristor-related non-idealities can be mitigated by optimizing the software-related design parameters for the neural network 259 . Reference 260 shows that neural network design parameters can be optimized to reduce the effects of conductance variations and conductance drift in memristors without compromising performance accuracy.…”

Section: Simulation Of Memristive Annsmentioning

confidence: 99%

Hardware implementation of memristor-based artificial neural networks

Aguirre,

Sebastian,

Le Gallo

et al. 2024

Nat Commun

Self Cite

View full text Add to dashboard Cite

Artificial Intelligence (AI) is currently experiencing a bloom driven by deep learning (DL) techniques, which rely on networks of connected simple computing units operating in parallel. The low communication bandwidth between memory and processing units in conventional von Neumann machines does not support the requirements of emerging applications that rely extensively on large sets of data. More recent computing paradigms, such as high parallelization and near-memory computing, help alleviate the data communication bottleneck to some extent, but paradigm- shifting concepts are required. Memristors, a novel beyond-complementary metal-oxide-semiconductor (CMOS) technology, are a promising choice for memory devices due to their unique intrinsic device-level properties, enabling both storing and computing with a small, massively-parallel footprint at low power. Theoretically, this directly translates to a major boost in energy efficiency and computational throughput, but various practical challenges remain. In this work we review the latest efforts for achieving hardware-based memristive artificial neural networks (ANNs), describing with detail the working principia of each block and the different design alternatives with their own advantages and disadvantages, as well as the tools required for accurate estimation of performance metrics. Ultimately, we aim to provide a comprehensive protocol of the materials and methods involved in memristive neural networks to those aiming to start working in this field and the experts looking for a holistic approach.

show abstract

Automating Analogue AI Chip Design with Genetic Search

Cited by 12 publications

References 46 publications

Analogue neuro-memristive convolutional dropout nets

Analogue neuro-memristive convolutional dropout nets

A Fully Integrated System‐on‐Chip Design with Scalable Resistive Random‐Access Memory Tile Design for Analog in‐Memory Computing

Hardware implementation of memristor-based artificial neural networks

Contact Info

Product

Resources

About