Memristor-only LSTM Acceleration with Non-linear Activation Functions - Designing Modern Embedded Systems: Software, Hardware, and Applications
Conference Papers Year : 2023

Memristor-only LSTM Acceleration with Non-linear Activation Functions

Abstract

Long Short-Term Memories (LSTMs) applied to Speech Recognition are an essential application of modern embedded devices. Computing Matrix-Vector Multiplications (MVMs) with Resistive Random Access Memory (ReRAM) crossbars has paved the way for solving the memory bottleneck issues related to LSTM processing. However, mixed-signal and fully-analog accelerators still lack in developing energy-efficient and versatile devices for the calculus of activation functions between MVM operations. This paper proposes a design methodology and circuitry that achieves both energy efficiency and versatility by introducing a programmable memristor array for computing nonlinearities. We exploit the inherent capability of ReRAM crossbars in computing MVM to perform piecewise linear interpolation (PWL) of non-linear activation functions, achieving a programmable device with a smaller cost. Experiments show that our approach outperforms state-of-the-art LSTM accelerators being 4.85x more efficient using representative speech recognition datasets.
Embargoed file
Embargoed file
1 0 10
Year Month Jours
Avant la publication
Thursday, January 1, 2026
Embargoed file
Thursday, January 1, 2026
Please log in to request access to the document

Dates and versions

hal-04645291 , version 1 (11-07-2024)

Licence

Identifiers

Cite

Rafael Moura, João Lima, Luigi Carro. Memristor-only LSTM Acceleration with Non-linear Activation Functions. 7th International Embedded Systems Symposium (IESS), Nov 2022, Lippstadt, Germany. pp.89-100, ⟨10.1007/978-3-031-34214-1_8⟩. ⟨hal-04645291⟩
27 View
1 Download

Altmetric

Share

More