3

I have a a list of training data that I am using to train. However, when I predict, the prediction will be done online with a single example at a time.

If I declare my model with input like the following

model = Sequential()
model.add(Dense(64, batch_input_shape=(100, 5, 1), activation='tanh'))
model.add(LSTM(32, stateful=True))
model.add(Dense(1, activation='linear'))
optimizer = SGD(lr=0.0005)
model.compile(loss='mean_squared_error', optimizer=optimizer)

When I go to predict with a single example of shape (1, 5, 1), it gives the following error.

ValueError: Shape mismatch: x has 100 rows but z has 1 rows

The solution I came up with was to just train my model iteratively using a batch_input_shape of (1,5,1) and calling fit for each single example. This is incredibly slow.

Is there not a way to train on a large batch size, but predict with a single example using LSTM?

Thanks for the help.

2 Answers 2

2

Try something like this:

model2 = Sequential()
model2.add(Dense(64, batch_input_shape=(1, 5, 1), activation='tanh')) 
model2.add(LSTM(32, stateful=True))
model2.add(Dense(1, activation='linear'))
optimizer2 = SGD(lr=0.0005)
model2.compile(loss='mean_squared_error', optimizer=optimizer)

for nb, layer in enumerate(model.layers):
    model2.layers[nb].set_weights(layer.get_weights())

You are simply rewritting weights from one model to another.

1
  • This seems to work. I think I tried this before, but I did it by calling get and set on the model which didn't work for some reason. I didn't try layer by layer. Thanks!
    – Lucas
    Apr 12, 2017 at 21:04
0

You have defined the input_shape in the first layer. Therefore sending a shape that does not match the preset-ed input_shape is in valid.

There are two way to achieve that: You can modify your model by changing batch_input_shape=(100, 5, 1) to input_shape=(5, 1) to avoid a preset-ed batch size. You can setup the batch_size=100 in model.fit().

Edit: Method 2

You define the exact same model as model2. Then model2.set_weights(model1.get_weights()).

If you want to use stateful==True, you actually want to use the hidden layers from the last batch as the initial states for the next batch. Therefore very batch size should be matched. Otherwise, you can just remove the stateful==True.

1
  • I have tried that. When using an LSTM layer with stateful=True it gives the error "If a RNN is stateful, a complete input_shape must be provided (including batch size)."
    – Lucas
    Apr 12, 2017 at 12:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.