基于文本的深度学习方法的TensorFlow实现(2)——使用RNN进行文本分类

使用RNN进行文本分类

开始输入管道

encoder = info.features['text'].encoder

该文本编码器以可逆的方式编码字符串，解码时返回其字节编码。

sample_string = 'Hello TensorFlow.' encoded_string = encoder.encode(sample_string) print ('Encoded string is {}'.format(encoded_string)) original_string = encoder.decode(encoded_string) print ('The original string: "{}"'.format(original_string))

准备训练数据

首先，创建编码字符串的批处理。

BUFFER_SIZE = 10000 BATCH_SIZE = 64 train_dataset = train_dataset.shuffle(BUFFER_SIZE) train_dataset = train_dataset.padded_batch(BATCH_SIZE, train_dataset.output_shapes) test_dataset = test_dataset.padded_batch(BATCH_SIZE, test_dataset.output_shapes)

创建模型

首先，建立顺序模型，从嵌入层开始。 tf.keras.layers.Bidirectional()与RNN一起使用，通过向前和向后传播输入，然后连接输出。 RNN通过迭代元素处理序列输入。

model = tf.keras.Sequential([ tf.keras.layers.Embedding(encoder.vocab_size, 64), tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)), tf.keras.layers.Dense(64, activation='relu'), tf.keras.layers.Dense(1, activation='sigmoid') ])

编译

model.compile(loss='binary_crossentropy', optimizer=tf.keras.optimizers.Adam(1e-4), metrics=['accuracy'])

训练模型

history = model.fit(train_dataset, epochs=10, validation_data=test_dataset, validation_steps=30) test_loss, test_acc = model.evaluate(test_dataset)

在填充序列上训练，在未填充序列上训练，可以导致偏斜。使用掩码避免这种情况。

def pad_to_size(vec, size): zeros = [0] * (size - len(vec)) vec.extend(zeros) return vec def sample_predict(sentence, pad): encoded_sample_pred_text = encoder.encode(sample_pred_text) if pad: encoded_sample_pred_text = pad_to_size(encoded_sample_pred_text, 64) encoded_sample_pred_text = tf.cast(encoded_sample_pred_text, tf.float32) predictions = model.predict(tf.expand_dims(encoded_sample_pred_text, 0)) return (predictions) # predict on a sample text without padding. sample_pred_text = ('The movie was cool. The animation and the graphics ' 'were out of this world. I would recommend this movie.') predictions = sample_predict(sample_pred_text, pad=False) print (predictions) # predict on a sample text with padding sample_pred_text = ('The movie was cool. The animation and the graphics ' 'were out of this world. I would recommend this movie.') predictions = sample_predict(sample_pred_text, pad=True) print (predictions)

堆叠两个或更多的LSTM层

Keras递归层有两种模式，通过return_sequences构造参数控制。

对于每个时间步长返回连续输出的完整序列 (batch_size, timesteps, output_features)仅返回每个输入序列的最后一个输出 (batch_size, output_features) model = tf.keras.Sequential([ tf.keras.layers.Embedding(encoder.vocab_size, 64), tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64, return_sequences=True)), tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)), tf.keras.layers.Dense(64, activation='relu'), tf.keras.layers.Dropout(0.5), tf.keras.layers.Dense(1, activation='sigmoid') ]) model.compile(loss='binary_crossentropy', optimizer=tf.keras.optimizers.Adam(1e-4), metrics=['accuracy']) history = model.fit(train_dataset, epochs=10, validation_data=test_dataset, validation_steps=30) test_loss, test_acc = model.evaluate(test_dataset) # predict on a sample text with padding sample_pred_text = ('The movie was not good. The animation and the graphics ' 'were terrible. I would not recommend this movie.') predictions = sample_predict(sample_pred_text, pad=True) print (predictions)

参考链接