0

I'm trying to create a stateful RNN in Keras were the input is a concatenation of embeddings and 3 integer inputs.

The following is the simplified version of the code and i added the dimensions of the layers as comments to make it easier to read

input = Input(batch_shape=(1,3,10))

inputTag = Lambda(lambda x: x[:,:,0:1])(input)
inputMeta =Lambda(lambda x: x[:,:,7:10])(input)

inputTag.shape, inputMeta.shape
#(TensorShape([Dimension(1), Dimension(3), Dimension(1)]), 
# TensorShape([Dimension(1), Dimension(3), Dimension(1), Dimension(3)]))

inputTagEnc = Embedding(tag_vocab_size,
                tag_emb_output_dim, 
                input_length = 1)(inputTag)

inputTagEnc.shape
#TensorShape([Dimension(1), Dimension(3), Dimension(1), Dimension(4)])

encodings =[inputTagEnc, inputMeta]
encodedInput = Concatenate()(encodings)

#Traceback (most recent call last):
#  File "<stdin>", line 1, in <module>
#  File "C:\Users\marco\Anaconda3\lib\site-packages\keras\engine\topology.py", line 521, in __call__
#    self.build(input_shapes)
#  File "C:\Users\marco\Anaconda3\lib\site-packages\keras\layers\merge.py", line 153, in build
#    'Got inputs shapes: %s' % (input_shape))
#ValueError: `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(1, 1, 4), (1, 3, 1, 3)]


rnn = LSTM(n_hidden, 
             dropout=0.0, 
             activation='relu',
             recurrent_dropout=0.0,             
             stateful=True)(encodedInput)
output = Dense(3, activation='softmax')(rnn)


model = Model(inputs=[input], outputs=[encodedInput])
model.compile(loss='categorical_crossentropy', optimizer=Adam())
model.summary()

The concatenation seems to be failing because the embedding is being unrolled as I have a batch size of 3, but the integer inputs are trying to processed as is.

Now I've tried a million variations of reshapes, flattens and Timedistributed trying to get the dimensions to align but I'm still stuck.

Does anyone have a solution to this problem?

1 Answer 1

0

Tumbleweed ...

I found a solution to this problem although not a very neat one. Using a custom layer I was able to stich the Tensors manually before feeding them into the RNN. The method needs some more work to make it dynamic but here it is in its current form.

class WorkingConcat(Layer):

def __init__(self, **kwargs):        
    super(WorkingConcat, self).__init__(**kwargs)

def build(self, input_shape):        
    super(WorkingConcat, self).build(input_shape)  # Be sure to call this somewhere!

def call(self, x):
    encodings, inputMeta = x[0:len(x)-1], x[len(x)-1]
    dd = K.reshape(inputMeta,(batch_size,3,1,3))
    encodings.append(dd)
    encodedInput = K.concatenate(encodings,axis=3)
    #ff = Reshape((3,67))(encodedInput)
    ff = K.reshape(encodedInput,(batch_size,3,67))
    self.output_dim = ff.shape
    return ff

def compute_output_shape(self, input_shape):
    return (batch_size, 3, 67)

encodings =[inputTagEnc]
encodings.append(inputMeta)

reshapedFrame = WorkingConcat()(encodings)

Custom layer can only take a tensor or list of tensors so the layer takes the last tensor it finds in the list and reshapes it and then concats it with the rest (in my full solution encodings has a bunch of encodings of similar sizes)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.