This study compares recurrent units in RNNs, focusing on LSTM and GRU. The models were evaluated on polyphonic music modeling and speech signal tasks. Results demonstrate that gating units outperform traditional tanh units, with GRU showing comparable performance to LSTM.
Recurrent neural networks (RNNs) excel in sequence-based tasks. This paper evaluates LSTM and GRU against traditional tanh units to understand their advantages in long-term dependencies and computational efficiency in various sequence modeling tasks.
The study uses three recurrent units: LSTM, GRU, and tanh. Models were trained on polyphonic music datasets and Ubisoft speech datasets. Parameters were balanced across models to ensure a fair comparison of efficiency, convergence speed, and accuracy.
GRU outperformed other models in polyphonic music tasks, except for Nottingham, and excelled in Ubisoft datasets. Both GRU and LSTM surpassed tanh in performance and convergence speed, highlighting the effectiveness of gating mechanisms in sequence modeling.
Gating units enable better handling of long-term dependencies and gradient stability. While LSTM offers controlled memory exposure, GRU simplifies computation. The choice between them may depend on the specific dataset and task requirements.
This paper confirms the superiority of gated recurrent units over traditional units in sequence modeling. Future research will focus on refining these units and exploring their performance across diverse datasets and tasks.