59

Ever since I learned about -j I've used -j8 blithely. The other day I was compiling an atlas installation and the make failed. Eventually I tracked it down to things being made out of order - and it worked fine once I went back to singlethreaded make. This makes me nervous. What sort of conditions do I need to watch for when writing my own make files to avoid doing something unexpected with make -j?

2
  • 6
    Are you sure it's make's fault? Writing correct makefiles is error-prone. Commented Oct 14, 2009 at 4:38
  • 1
    Meh, even makefiles generated by autotools are buggy. Try compiling GCC with -j2 or up. Commented Oct 14, 2009 at 4:46

6 Answers 6

50

I think make -j will respect the dependencies you specify in your Makefile; i.e. if you specify that objA depends on objB and objC, then make won't start working on objA until objB and objC are complete.

Most likely your Makefile isn't specifying the necessary order of operations strictly enough, and it's just luck that it happens to work for you in the single-threaded case.

Sign up to request clarification or add additional context in comments.

5 Comments

That's correct. I work on a code base of about 20 million lines, mostly in C with a little C++. It's split into hundreds of components, about half of which use make, and half of which use jam. I always do parallel compiles with the -j option; otherwise, builds would take hours. Jam generates its own dependencies, so the components that use it always succeed. But components that use hand-built makefiles choke on occasion, invariably due to inadequate dependencies.
I've seen a lot on the web that -j is referring to how many CPU's you want to compile with. I've run a test on my 8 core machine and make works up to and including -j8 (beyond that is 100% usage on all cores, similar to -j8). At each integer increment I would see another core usage go up by roughly 100%. Would you say that is a good rule of thumb for knowing what to specify as your -j value?
The only good rule of thumb I know is to do a "make clean ; time make -j n" on your program, with various values of (n), and see how long each build takes, and then go with whichever setting completed in the least amount of time. There are too many variables (CPU speed, RAM speed, RAM size, hard drive speed, OS process overhead, etc) to be able to make useful generalizations.
If you use the -j parameter without providing any integer value, make will use the system cpu cores without any limitation which is equal to the highest number of processor count in /proc/cpuinfo.
I like the use of time, but some of you may find make -j [N] --debug=j useful, possibly along with time. This adds job information to debugging, including added/active/reaped targets with respective PID.
27

In short - make sure that your dependencies are correct and complete.

If you are using a single threaded make then you can be blindly ignoring implicit dependencies between targets. When using parallel make you can't rely on the implicit dependencies. They should all be made explicit. This is probably the most common trap. Particularly if using .phony targets as dependencies.

This link is a good primer on some of the issues with parallel make.

1 Comment

+1 For the good link. I now feel like i can trust make -j too and how to fix problems when they arise. Well worth reading.
12

Here's an example of a problem that I ran into when I started using parallel builds. I have a target called "fresh" that I use to rebuild the target from scratch (a "fresh" build). In the past, I coded the "fresh" target by simply indicating "clean" and then "build" as dependencies.

build: ## builds the default target
clean: ## removes generated files
fresh: clean build ## works for -j1 but fails for -j2

That worked fine until I started using parallel builds, but with parallel builds, it attempts to do both "clean" and "build" simultaneously. So I changed the definition of "fresh" as follows in order to guarantee the correct order of operations.

fresh:
    $(MAKE) clean
    $(MAKE) build

This is fundamentally just a matter of specifying dependencies correctly. The trick is that parallel builds are more strict about this than are single-threaded builds. My example demonstrates that a list of dependencies for given target does not necessarily indicate the order of execution.

6 Comments

Recursive make, yuk!. The correct way to say to make please always do clean before build is of course build: clean.
@bobbogo: I don't want to always do clean before build -- that is unnecessary in most cases. The "fresh" target that I described is basically a just a simple script that runs a "make clean" followed by a "make build" (for those rare times when I want to do that). I don't see any harm in executing make recursively in this scenario.
How about protecting it with a conditional: ifeq ($(MAKECMDGOALS), fresh); build: clean; endif (replace semicolons by newlines)?
The conditional is a step in the right direction, but I'd recommend auto-dependency generation any day. No need for recursive make OR cleaning. If I've changed any files referenced by my target or any prerequisites of my target, make will create a directed acyclic graph of everything that must be rebuilt, allowing any make -j N to succeed (including just make -j.)
@JohnP: I absolutely agree with auto-dependency generation, but that doesn't mean you won't want to have a clean target for special occasions. Also of note, I found that the use of auto-dependency generation necessitated a recursive make because otherwise a corrupt dependency list can completely disable your makefile (including the clean target). One way a dependency list could become corrupt is if you Ctrl-C during the build.
|
7

If you have a recursive make, things can break pretty easily. If you're not doing a recursive make, then as long as your dependencies are correct and complete, you shouldn't run into any problems (save for a bug in make). See Recursive Make Considered Harmful for a much more thorough description of the problems with recursive make.

1 Comment

> "To avoid the symptoms, it is only necessary to avoid the separation; to use a single Makefile for the whole project." Seriously ... I guess these guys never had to work in a real production environment.
3

It is a good idea to have an automated test to test the -j option of ALL the make files. Even the best developers have problems with the -j option of make. The most common issues is the simplest.

myrule: subrule1 subrule2
     echo done

subrule1:
     echo hello

subrule2:
     echo world

In normal make, you will see hello -> world -> done. With make -j 4, you will might see world -> hello -> done

Where I have see this happen most is with the creation of output directories. For example:

build: $(DIRS) $(OBJECTS)
     echo done

$(DIRS):
     -@mkdir -p $@

$(OBJECTS):
     $(CC) ...

Comments

1

Just thought I would add to subsetbrew's answer as it does not show the effect clearly. However adding some sleep commands does. Well it works on linux.

Then running make shows differences with:

  • make
  • make -j4

all: toprule1

toprule1: botrule2 subrule1 subrule2
    @echo toprule 1 start
    @sleep 0.01
    @echo toprule 1 done

subrule1: botrule1
    @echo subrule 1 start
    @sleep 0.08
    @echo subrule 1 done

subrule2: botrule1
    @echo subrule 2 start
    @sleep 0.05
    @echo subrule 2 done

botrule1:
    @echo botrule 1 start
    @sleep 0.20
    @echo "botrule 1 done (good prerequiste in sub)"

botrule2:
    @echo "botrule 2 start"
    @sleep 0.30
    @echo "botrule 2 done (bad prerequiste in top)"

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.