基于keras的双层LSTM网络和双向LSTM网络
1 前言基于keras的双层LSTM网络和双向LSTM网络中,都会用到 LSTM层,主要参数如下:LSTM(units,input_shape,return_sequences=False)units:隐藏层神经元个数input_shape=(time_step, input_feature):time_step是序列递归的步数,input_feature是输入特征维数re...
1 前言
基于keras的双层LSTM网络和双向LSTM网络中,都会用到 LSTM层,主要参数如下:
LSTM(units,input_shape,return_sequences=False)
- units:隐藏层神经元个数
- input_shape=(time_step, input_feature):time_step是序列递归的步数,input_feature是输入特征维数
- return_sequences: 取值为True,表示每个时间步的值都返回;取值为False,表示只返回最后一个时间步的取值
本文以MNIST手写数字分类为例,讲解双层LSTM网络和双向LSTM网络的实现。关于MNIST数据集的说明,见使用TensorFlow实现MNIST数据集分类。
笔者工作空间如下:
代码资源见--> 双隐层LSTM和双向LSTM
2 双层LSTM网络
![](https://i-blog.csdnimg.cn/blog_migrate/51095a902770cd9493a5805f74f1384c.png)
DoubleLSTM.py
from tensorflow.examples.tutorials.mnist import input_data
from keras.models import Sequential
from keras.layers import Dense,LSTM
#载入数据
def read_data(path):
mnist=input_data.read_data_sets(path,one_hot=True)
train_x,train_y=mnist.train.images.reshape(-1,28,28),mnist.train.labels,
valid_x,valid_y=mnist.validation.images.reshape(-1,28,28),mnist.validation.labels,
test_x,test_y=mnist.test.images.reshape(-1,28,28),mnist.test.labels
return train_x,train_y,valid_x,valid_y,test_x,test_y
#双层LSTM模型
def DoubleLSTM(train_x,train_y,valid_x,valid_y,test_x,test_y):
#创建模型
model=Sequential()
model.add(LSTM(64,input_shape=(28,28),return_sequences=True)) #返回所有节点的输出
model.add(LSTM(32,return_sequences=False)) #返回最后一个节点的输出
model.add(Dense(10,activation='softmax'))
#查看网络结构
model.summary()
#编译模型
model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])
#训练模型
model.fit(train_x,train_y,batch_size=500,nb_epoch=15,verbose=2,validation_data=(valid_x,valid_y))
#评估模型
pre=model.evaluate(test_x,test_y,batch_size=500,verbose=2)
print('test_loss:',pre[0],'- test_acc:',pre[1])
train_x,train_y,valid_x,valid_y,test_x,test_y=read_data('MNIST_data')
DoubleLSTM(train_x,train_y,valid_x,valid_y,test_x,test_y)
每层网络输出尺寸:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
lstm_5 (LSTM) (None, 28, 64) 23808
_________________________________________________________________
lstm_6 (LSTM) (None, 32) 12416
_________________________________________________________________
dense_5 (Dense) (None, 10) 330
=================================================================
Total params: 36,554
Trainable params: 36,554
Non-trainable params: 0
由于第一个LSTM层设置了 return_sequences=True,每个节点的输出值都会返回,因此输出尺寸为 (None, 28, 64)
由于第二个LSTM层设置了 return_sequences=False,只有最后一个节点的输出值会返回,因此输出尺寸为 (None, 32)
训练结果:
Epoch 13/15
- 17s - loss: 0.0684 - acc: 0.9796 - val_loss: 0.0723 - val_acc: 0.9792
Epoch 14/15
- 18s - loss: 0.0633 - acc: 0.9811 - val_loss: 0.0659 - val_acc: 0.9822
Epoch 15/15
- 17s - loss: 0.0597 - acc: 0.9821 - val_loss: 0.0670 - val_acc: 0.9812
test_loss: 0.0714278114028275 - test_acc: 0.9789000034332276
3 双向LSTM网络
![](https://i-blog.csdnimg.cn/blog_migrate/e84f7db1de26c261d4ad1e1ee9329be0.png)
from tensorflow.examples.tutorials.mnist import input_data
from keras.models import Sequential
from keras.layers import Dense,LSTM,Bidirectional
#载入数据
def read_data(path):
mnist=input_data.read_data_sets(path,one_hot=True)
train_x,train_y=mnist.train.images.reshape(-1,28,28),mnist.train.labels,
valid_x,valid_y=mnist.validation.images.reshape(-1,28,28),mnist.validation.labels,
test_x,test_y=mnist.test.images.reshape(-1,28,28),mnist.test.labels
return train_x,train_y,valid_x,valid_y,test_x,test_y
#双向LSTM模型
def BiLSTM(train_x,train_y,valid_x,valid_y,test_x,test_y):
#创建模型
model=Sequential()
lstm=LSTM(64,input_shape=(28,28),return_sequences=False) #返回最后一个节点的输出
model.add(Bidirectional(lstm)) #双向LSTM
model.add(Dense(10,activation='softmax'))
#编译模型
model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])
#训练模型
model.fit(train_x,train_y,batch_size=500,nb_epoch=15,verbose=2,validation_data=(valid_x,valid_y))
#查看网络结构
model.summary()
#评估模型
pre=model.evaluate(test_x,test_y,batch_size=500,verbose=2)
print('test_loss:',pre[0],'- test_acc:',pre[1])
train_x,train_y,valid_x,valid_y,test_x,test_y=read_data('MNIST_data')
BiLSTM(train_x,train_y,valid_x,valid_y,test_x,test_y)
每层网络输出尺寸:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
bidirectional_5 (Bidirection (None, 128) 47616
_________________________________________________________________
dense_6 (Dense) (None, 10) 1290
=================================================================
Total params: 48,906
Trainable params: 48,906
Non-trainable params: 0
由于LSTM层设置了 return_sequences=False,只有最后一个节点的输出值会返回,每层LSTM返回64维向量,两层合并共128维,因此输出尺寸为 (None, 128)
训练结果:
Epoch 13/15
- 22s - loss: 0.0512 - acc: 0.9839 - val_loss: 0.0632 - val_acc: 0.9790
Epoch 14/15
- 22s - loss: 0.0453 - acc: 0.9865 - val_loss: 0.0534 - val_acc: 0.9832
Epoch 15/15
- 22s - loss: 0.0418 - acc: 0.9869 - val_loss: 0.0527 - val_acc: 0.9830
test_loss: 0.06457789749838412 - test_acc: 0.9795000076293945
![Logo](https://devpress.csdnimg.cn/79de2bf0b7994defa4242ef90d5513fa.jpg)
开放原子开发者工作坊旨在鼓励更多人参与开源活动,与志同道合的开发者们相互交流开发经验、分享开发心得、获取前沿技术趋势。工作坊有多种形式的开发者活动,如meetup、训练营等,主打技术交流,干货满满,真诚地邀请各位开发者共同参与!
更多推荐
所有评论(0)