I've compared LSTM result with Keras/Tensorflow calculation and Numpy calculation. However, the result is slightly different:
Numpy: [[ 0.16315128 -0.04277606 0.26504123 0.08014129 0.38561829]]
Keras: [[ 0.16836338 -0.04930305 0.25080156 0.08938988 0.3537751 ]]
Keras' LSTM implementation does not use tf.contrib.rnn
but Keras directly manages the parameters, and tf.matmul
is used to calculate. I found the corresponding implementation of Keras and tried the same calculation with Numpy, but the values are slightly different as shown above.
I have checked the formula several times and it seems like the same. The only difference is the differences between tf.matmul
or np.dot
. Maybe there are some differences about decimal point calculation method. Even so, I think the results are too much different. The biggest difference is about 10%. I'd like to match the Numpy calculation with the tensorflow calculation. If someone could give me some hint or point me to the right implementation, I'd really appreciate it.
Keras implementation and the Numpy code implemented myself: