One target in my makefile is a very CPU and time consuming task. But I can split the workload and run the task several times in parallel to speed up the entire process.
My problem is that make doesn't wait for all processes to complete.
Consider this simple script, named myTask.sh
:
#!/bin/bash
echo "Sleeping $1 seconds"
sleep $1
echo "$1 are over!"
Now, let's call this from a bash script, and use wait
to wait for all tasks to complete:
#!/bin/bash
echo "START"
./myTask.sh 5 &
./myTask.sh 15 &
./myTask.sh 10 &
wait # Wait for all tasks to complete
echo "DONE"
The output is as expected:
START
Sleeping 15 seconds
Sleeping 5 seconds
Sleeping 10 seconds
5 are over!
10 are over!
15 are over!
DONE
But when trying the same in a Makefile
:
test:
echo "START"
./myTask.sh 5 &
./myTask.sh 15 &
./myTask.sh 10 &
wait
echo "DONE"
it doesn't work:
START
Sleeping 5 seconds
Sleeping 15 seconds
Sleeping 10 seconds
DONE
sweber@pc:~/testwait $5 are over!
10 are over!
15 are over!
Of course, I could create multiple targets which can be "built" in parallel by make, or let make just run the bash script which runs the tasks. But is there a way to do it more like what I already tried?
./myTask.sh 5 & ./myTask.sh 15 & ./myTask.sh 10 & wait
parallel ./myTask.sh ::: 5 10 15
ONESHELL
feature is installed, your script will work "as-is" if you add a line with.ONESHELL:
at the start since that will force your entire recipe to run in a single invocation of your shell.