Member-only story
Multi Class Text Classification with Keras and LSTM
In this tutorial, we will build a text classification with Keras and LSTM to predict the category of the BBC News articles.
LSTM (Long Short Term Memory)
LSTM was designed to overcome the problems of simple Recurrent Network (RNN) by allowing the network to store data in a sort of memory that it can access at a later times. LSTM is a special type of Recurrent Neural Network (RNN) that can learn long term patterns.
The key to LSTMs is the cell state, the horizontal line running through the top of the diagram. The cell state is updated twice with few computations that resulting stabilize gradients. It has also a hidden state that acts like a short term memory.
In LSTM there are Forget Gate, Input Gate and Output Gate that we will walk through it shortly.
The first step is to decide what information we’re going to throw away from the cell state. This decision is made by a sigmoid layer called the “Forget Gate layer.”
The next step is to decide what new information we’re going to store in the cell state. This has two parts. First, a sigmoid layer called the “Input Gate layer” decides which values we’ll update. Next, a tanh layer creates a vector of new candidate values that could be added to the state.
In the next step, we’ll combine these two to create an update to the cell state.
Then we update the old cell state into the new cell state.
