I'm trying to batch some python scripts that train deep neural networks using Keras and I'm encountering the problem that when the first python script finishes, the whole batch is stopped.
I tried writing a python wrapper, but still nothing changes. I wonder if this problem is OS related or it has to do with Keras framework or python.
The first attempt was to write a shell script:
#!/bin/sh
for i in {1..10}; do python my_experiment.py; done
However, after running my_experiment.py
once, the whole batch stops.
I tried using Python:
import subprocess
import os
def main():
for i in xrange(10):
subprocess.call(['python', os.path.join(os.getcwd(), 'my_experiment.py')])
if __name__ == "__main__":
main()
But I still get the same result.
Has anyone encountered this problem before? If not I would welcome ideas to try and figure out what's going on.
Additionally, I'm running this process in a remote machine via SSH, in the following way:
$ ./batch_script.sh &
$ disown
Or in the case of python:
$ python batch_script.py &
$ disown
Cheers