Efficient implementation of piecewise linear activation function for digital VLSI neural networks

Myers, David Jaz; Hutchinson, Rebecca

doi:10.1049/el:19891114

Cited by 65 publications

(25 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach gives a better maximum error than both the first and second order approximation of [11]. [8] [ -8,8) N/A 0.0490 0.0247 Alippi et al [9] [ -8,8) N/A 0.0189 0.0087 Amin et al [10] [ [12] [-5,5] 5 0.0050 n/a Basterretxea et al (q=3) [13] [ -8,8) N/A 0.0222 0.0077 Tommiska (337) [14] [ -8,8) N/A 0.0039 0.0017 Tommiska (336) [14] [ -8,8) N/A 0.0077 0.0033 Tommiska (236) [14] [-4,4) N/A 0.0077 0.0040 Tommiska (235) [14] [ …”

Section: Resultsmentioning

confidence: 93%

“…Furthermore, there is considerable variance within each category. For example, an A-Law companding technique is used in [8], a sum of steps approximation is used in [9], a multiplier-less piecewise approximation is presented in [10] and a recursive piecewise multiplier-less approximation is presented in [13]. An elementary function generator capable of multiple activation functions using a first and second order polynomial approximation is detailed in [11].…”

Section: Related Researchmentioning

confidence: 99%

See 1 more Smart Citation

An Efficient Hardware Architecture for a Neural Network Activation Function Generator

Larkin

Kinane

Mureşan

et al. 2006

Advances in Neural Networks - ISNN 2006

View full text Add to dashboard Cite

Abstract. This paper proposes an efficient hardware architecture for a function generator suitable for an artificial neural network (ANN). A spline-based approximation function is designed that provides a good trade-off between accuracy and silicon area, whilst also being inherently scalable and adaptable for numerous activation functions. This has been achieved by using a minimax polynomial and through optimal placement of the approximating polynomials based on the results of a genetic algorithm. The approximation error of the proposed method compares favourably to all related research in this field. Efficient hardware multiplication circuitry is used in the implementation, which reduces the area overhead and increases the throughput.

show abstract

Section: Resultsmentioning

confidence: 93%

Section: Related Researchmentioning

confidence: 99%

An Efficient Hardware Architecture for a Neural Network Activation Function Generator

Larkin

Kinane

Mureşan

et al. 2006

Advances in Neural Networks - ISNN 2006

View full text Add to dashboard Cite

show abstract

“…The implementation of the neuron's nonlinear activation function and their derivatives used by the learning algorithm, is often solved by a piecewise linear approximation [4,5,7,[17][18][19][20] . However, no implementation method has emerged as a universal solution.…”

Section: Simulation Resultsmentioning

confidence: 99%

Digital Hardware Implementation of a Neural System Used for Nonlinear Adaptive Prediction

Faiedh¹,

Souani²,

Torki³

et al. 2006

J. of Computer Science

View full text Add to dashboard Cite

Neural networks have been widely used for many applications in digital communications. They are able to give solutions to complex problems due to their nonlinear processing and their learning and generalization. Neural networks are one of the key technologies for the communication domain and accordingly a special effort may be expected to be paid to real time hardware implementation issues. In this study, it is proposed a digital hardware implementation of a neural system based on a multilayer perceptron (MLP). The neural system is used for the nonlinear adaptive prediction of nonstationary signals such as speech signals. The implemented architecture of the MLP is generated using a generic elementary neuron (EN). The polynomial approximation method is used to implement the sigmoidal activation function. The back-propagation algorithm is used to implant the prediction task. The circuit implementation architecture is detailed, for achieving real-time prediction for speech signals. The designed ASIC circuit includes a neural network block, an on-chip learning block and a memory used for storing the synaptic weights for updating.

show abstract

“…Mathematical approximations of membership functions [5], [6] are also widely used but any errors produced by crude approximations can be problematic for adaptive systems as a continuous function approximation may not prove to be continuous in all segments.…”

Section: A Fuzzificationmentioning

confidence: 99%

Parallel Type-2 Fuzzy Logic Co-Processors for Engine Management

Lynch

Hagras

Callaghan

2007

2007 IEEE International Fuzzy Systems Conference

View full text Add to dashboard Cite

Abstract-Marine diesel engines operate in highly dynamic and uncertain environments, hence they require robust and accurate speed controllers that can handle the encountered uncertainties. Type-2 Fuzzy Logic Controllers (FLCs) can handle such uncertainties; however there are a number of computational bottlenecks posing as significant barriers, preventing widespread deployment of type-2 FLCs in commercial embedded control systems. This paper explores the use of parallel hardware implementations of interval type-2 FLC as a means to eradicate these barriers producing bespoke co-processors for a soft core implementation of a FPGA based 32 bit RISC micro-processor. These co-processors will perform functions such as fuzzification and type reduction and are currently utilised as part of a larger embedded Interval Type-2 Fuzzy Engine Management System (T2FEMS). Numerous timing comparisons were undertaken between the coprocessors and hard coded type reducers were the type-2 coprocessors reduced the required computational cycles by 99.88 percent. Thus the co-processors enable us to fully explore the potential of interval and potentially general type-2 systems in applied commercial embedded applications.

show abstract

Efficient implementation of piecewise linear activation function for digital VLSI neural networks

Cited by 65 publications

References 2 publications

An Efficient Hardware Architecture for a Neural Network Activation Function Generator

An Efficient Hardware Architecture for a Neural Network Activation Function Generator

Digital Hardware Implementation of a Neural System Used for Nonlinear Adaptive Prediction

Parallel Type-2 Fuzzy Logic Co-Processors for Engine Management

Contact Info

Product

Resources

About