Precise digital implementations of hyperbolic tanh and sigmoid function

Gomar, Shaghayegh; Mirhassani, Mitra; Ahmadi, M.

doi:10.1109/acssc.2016.7869646

Cited by 59 publications

(18 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Approximations of sigmoid and hyperbolic tangent functions have been presented in many studies, as shown in [34][35][36]. In this work, the piecewise-linear (PWL) approximation method is adopted to implement the hyperbolic tangent and sigmoid activation function because of high approximation accuracy with the adjustable number of lines and simplicity of hardware implementation.…”

Section: Stage 2: Configurable High-performance Low-power Mlp Microarchitecturementioning

confidence: 99%

AHEAD: Automatic Holistic Energy-Aware Design Methodology for MLP Neural Network Hardware Generation in Proactive BMI Edge Devices

et al. 2020

View full text Add to dashboard Cite

The prediction of a high-level cognitive function based on a proactive brain–machine interface (BMI) control edge device is an emerging technology for improving the quality of life for disabled people. However, maintaining the stability of multiunit neural recordings is made difficult by the nonstationary nature of neurons and can affect the overall performance of proactive BMI control. Thus, it requires regular recalibration to retrain a neural network decoder for proactive control. However, retraining may lead to changes in the network parameters, such as the network topology. In terms of the hardware implementation of the neural decoder for real-time and low-power processing, it takes time to modify or redesign the hardware accelerator. Consequently, handling the engineering change of the low-power hardware design requires substantial human resources and time. To address this design challenge, this work proposes AHEAD: an automatic holistic energy-aware design methodology for multilayer perceptron (MLP) neural network hardware generation in proactive BMI edge devices. By taking a holistic analysis of the proactive BMI design flow, the approach makes judicious use of the intelligent bit-width identification (BWID) and configurable hardware generation, which autonomously integrate to generate the low-power hardware decoder. The proposed AHEAD methodology begins with the trained MLP parameters and golden datasets and produces an efficient hardware design in terms of performance, power, and area (PPA) with the least loss of accuracy. The results show that the proposed methodology is up to a 4X faster in performance, 3X lower in terms of power consumption, and achieves a 5X reduction in area resources, with exact accuracy, compared to floating-point and half-floating-point design on a field-programmable gate array (FPGA), which makes it a promising design methodology for proactive BMI edge devices.

show abstract

Section: Stage 2: Configurable High-performance Low-power Mlp Microarchitecturementioning

confidence: 99%

AHEAD: Automatic Holistic Energy-Aware Design Methodology for MLP Neural Network Hardware Generation in Proactive BMI Edge Devices

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The weight parameter matrix and offset parameter vector of the kth, k = 1, 2, … , K, hidden layer of the compression encoder are denoted by E w, k and E b, k , respectively. The first K − 2 hidden layers use sigmoid activation function, 35 which is expressed as The last two hidden layers adopt tanh activation function, 35 which is expressed as…”

Section: Structure and Working Principle Of The Sending Endmentioning

confidence: 99%

Adaptive modem and interference suppression based on deep learning

Wei

Wang

Luo

2021

Trans Emerging Tel Tech

View full text Add to dashboard Cite

With the increasingly fierce competition of electromagnetic spectrum, developing intelligent communication systems that can reconfigure its waveform can effectively improve the communication system's ability to adapt to the complex electromagnetic environment. In this paper, an adaptive modem based on a deep autoencoder network (DAN) is designed, and the demodulation performance that is close to, consistent with, or better than traditional MPSK or QAM is achieved. This DAN can be trained by a unified loss function and a unified optimization algorithm to implement a variety of modems. For high-order modems, the constellation diagrams generated by the DAN are radically different from the traditional modulation and very difficult to distinguish linearly, which is beneficial to improve the anti-interception ability of the communication systems. Additionally, based on the trained DAN, a convolutional neural network (CNN) with a single convolutional layer is incorporated to suppress a variety of interferences. By using transfer learning, the new deep learning (DL) model with the CNN converges fast with a few epochs of training. Training and test results verify that the new DL model can improve the anti-interference ability of the communication systems by learning and suppressing the interferences in both frequency and power domains.

show abstract

“…where g(•) is the activation function of neurons, N m is the combination of input characteristic graphs, ⊗ represents convolution operation, k n−1 rm is the convolution kernel matrix, and b r m is the bias matrix. The activation functions commonly used in neural networks include sigmoid function [12], ReLU function [13], tanh function [14], etc.…”

Section: A Convolution Neural Networkmentioning

confidence: 99%

Distinguishing Oracle Variants Based on the Isomorphism and Symmetry Invariances of Oracle-Bone Inscriptions

Gao

Liang

2020

IEEE Access

View full text Add to dashboard Cite

Oracle bone inscriptions are the earliest mature writing system discovered in China. Oracle is usually published in the form of glyphs and pictures, and the related oracle font database environment is closed and limited. Due to the lack of standardization of ancient oracle bone inscriptions, the structure and glyph of the same characters are not unified, and a large number of different shapes and complicated characters coexist. Even ancient character devotees need to complete the basic recognition of oracle with the help of professional reference books and experts. In addition, in the field of traditional literature, distinguishing variant characters of oracle-bone inscriptions needs strong expert knowledge and lacks efficiency. The recognition process consists of two stages. According to the characteristics of oraclebone inscriptions, such as left-right symmetry, up-down symmetry and positive and negative coexistence, the corresponding data enhancement methods are used to get sufficient training samples in the first stage. In the second stage, the recognition candidate set under each threshold condition in the first stage are used to select the recognition candidate set which is empty under the specific threshold condition, and introduce the prior knowledge to distinguish the oracle variants. In the first stage, computer related methods are used to identify oracle variants. In the second stage, on the basis of the first stage, a set of recognition results that cannot be determined roughly is selected, and a priori knowledge is introduced to integrate multi-domain methods to identify oracle variants. The method proposed in this paper has achieved great results in the recognition of oracle variant characters.INDEX TERMS Oracle-bone inscriptions, pattern recognition, transfer learning, recognition candidate set.

show abstract

Precise digital implementations of hyperbolic tanh and sigmoid function

Cited by 59 publications

References 9 publications

AHEAD: Automatic Holistic Energy-Aware Design Methodology for MLP Neural Network Hardware Generation in Proactive BMI Edge Devices

AHEAD: Automatic Holistic Energy-Aware Design Methodology for MLP Neural Network Hardware Generation in Proactive BMI Edge Devices

Adaptive modem and interference suppression based on deep learning

Distinguishing Oracle Variants Based on the Isomorphism and Symmetry Invariances of Oracle-Bone Inscriptions

Contact Info

Product

Resources

About