07 Oct Long Short Term Memory (LSTM) – Deep Learning Model

The Long Short-Term Memory (LSTM) Model is designed to capture long-term dependencies in sequential data. It is a type of RNN. LSTMs easily store information about past inputs using a cell state. The network makes predictions about the current input since the cell state gets updated at each step.

Note: The core of the LSTM holds the cell state.

Process of LSTM

LSTM includes removing or adding information to a cell state. The gates are the structures that monitor this process. These gates let information pass through. Also, the core of the LSTM holds the cell state.

• Input Sequence: This is the input that is fed into the network one step at a time.
• Forget Gate: It decides the information to discard from the cell state. Formulae:
• Input Gate: Determines the new information to add to the cell state.
• Cell State Update: It combines the old cell state and new information.
• Output Gate: It decides which information from the cell state to output.

Advantages of LSTM

The following are the advantages of LSTM:

• LSTM remembers information for long periods. For tasks with sequences, the LSTMs are the most suitable.
• It is widely used for language modeling, time-series prediction, speech recognition, etc.
• It can easily capture patterns in sequential data, such as text, audio, time-series data, etc.
• LSTMs are adaptive to various types of data. This means it can learn which is the key information and should be remembered, or forgotten, etc. LSTM includes removing or adding information to a cell state

Disadvantages of LSTM

The following are the disadvantages of LSTM:

• LSTMs are more complex than traditional RNNs
• Training LSTMs can be computationally expensive and time-consuming
• Training LSTMs require substantial computational resources.
• LSTMs often require large amounts of training data to perform well.
• LSTMs can overfit the training data, especially when the dataset is small if proper regularization techniques are not used.

Applications of LSTM

• Language Modeling and Translation: Predicting sequences of words, sentences, or translations.
• Speech Recognition: Converting spoken language into text.
• Time Series Forecasting: Predicting future values in a time series, such as stock prices or weather data.
• Video Analysis: Understanding and predicting sequences of video frames.

Example of LSTM

Let us see an example to predict the next word in a Sentence “Hope you are doing”:

1. Input Sequence: “Hope you are doing”
2. Forget Gate: Decides which parts of the previous context are irrelevant and should be forgotten.
3. Input Gate: Decides which parts of the current input should be added to the cell state.
4. Cell State Update: Combines the old cell state (filtered by the forget gate) with the new candidate values (scaled by the input gate).
5. Output Gate: Decides what the next hidden state should be, using the current cell state and input.
6. Output: Predicts “well” based on the input sequence and updated cell state.

If you liked the tutorial, spread the word and share the link and our website Studyopedia with others.

For Videos, Join Our YouTube Channel: Join Now

Read More:

• Feedforward Neural Networks (FNN)
• Convolutional Neural Network (CNN)
• Recurrent Neural Networks (RNN)
• What is Deep Learning
• Deep Learning – Features
• AI vs ML vs DL
• Deep Learning – Advantages and Disadvantages
• Deep Learning- Applications
• Deep Learning – Types
• Neural Network