python – 嵌入在tensorflow中做了什么

我无法弄清楚嵌入和embedding_lookup在这里做了什么.它如何为张量添加另一个维度？从(20,25)到(20,25,200).在这种情况下(20,25)是20的批量大小,具有25个时间步长.我不明白为什么/为什么你可以添加单元格的hidden_??size作为输入数据的维度？通常,输入数据将是大小为[batch_size,num_features]的矩阵,并且模型将映射num_features —> hidden_??dims,其大小为[num_features,hidden_??dims]矩阵,产生的大小为[batch-size,hidden-dims].那么hidden_??dims如何成为输入张量的维度呢？

input_data,targets = reader.ptb_producer(train_data,20,25)
cell = tf.nn.rnn_cell.BasicLSTMCell(200,forget_bias=1.0,state_is_tuple=True)
initial_state = cell.zero_state(20,tf.float32)
embedding = tf.get_variable("embedding",[10000,200],dtype=tf.float32)
inputs = tf.nn.embedding_lookup(embedding,input_data)

input_data_train # <tf.Tensor 'PTBProducer/Slice:0' shape=(20,25) dtype=int32>
inputs # <tf.Tensor 'embedding_lookup:0' shape=(20,200) dtype=float32>

outputs = []
state = initial_state
for time_step in range(25):
    if time_step > 0: 
        tf.get_variable_scope().reuse_variables()

    cell_output,state = cell(inputs[:,time_step,:],state)
    outputs.append(cell_output)

output = tf.reshape(tf.concat(1,outputs),[-1,200])

outputs # list of 20: <tf.Tensor 'BasicLSTMCell/mul_2:0' shape=(20,200) dtype=float32>
output # <tf.Tensor 'Reshape_2:0' shape=(500,200) dtype=float32>

softmax_w = tf.get_variable("softmax_w",[config.hidden_size,config.vocab_size],dtype=tf.float32)
softmax_b = tf.get_variable("softmax_b",dtype=tf.float32)
logits = tf.matmul(output,softmax_w) + softmax_b

loss = tf.nn.seq2seq.sequence_loss_by_example([logits],[tf.reshape(targets,[-1])],[tf.ones([20*25],dtype=tf.float32)])
cost = tf.reduce_sum(loss) / batch_size