python – 在窗口分类上使用Tensorflow时嵌入向量未更新

单词嵌入矩阵称为word_vec并随机初始化(我也试过Xavier).

并且ind变量是来自矩阵的词向量的索引的向量.

第一层是config [‘window_size’](5)连接的单词向量.

word_vecs = tf.Variable(tf.random_uniform([len(words),config['embed_size']],-1.0,1.0),dtype=tf.float32)
ind = tf.placeholder(tf.int32,[None,config['window_size']])
x = tf.concat(1,tf.unpack(tf.nn.embedding_lookup(word_vecs,ind),axis=1))
W0 = tf.Variable(tf.random_uniform([config['window_size']*config['embed_size'],config['hidden_layer']]))
b0 = tf.Variable(tf.zeros([config['hidden_layer']]))
W1 = tf.Variable(tf.random_uniform([config['hidden_layer'],out_layer]))
b1 = tf.Variable(tf.zeros([out_layer]))
y0 = tf.nn.tanh(tf.matmul(x,W0) + b0)
y1 = tf.nn.softmax(tf.matmul(y0,W1) + b1)
y_ = tf.placeholder(tf.float32,out_layer])
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y1),reduction_indices=[1]))
train_step = tf.train.AdamOptimizer(0.5).minimize(cross_entropy)

这就是我运行图表的方式：

init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
for i in range(config['iterations'] ):
    r = random.randint(0,len(sentences)-1)
    inds=generate_windows([w for w,t in sentences[r]])
    #inds now contains an array of n rows on window_size columns
    ys=[one_hot(tags.index(t),len(tags)) for w,t in sentences[r]]
    #ys now contains an array of n rows on output_size columns
    sess.run(train_step,feed_dict={ind: inds,y_: ys})

维度工作,代码运行

但是,准确度几乎为零,我怀疑单词向量没有正确更新.

如何让tensorflow从连接窗口形式更新单词向量？