26

I have two development branches in git and I frequently need to change between the two. However, the really frustrating thing is that every time I change branches in git, the entire project gets rebuilt because the file-system timestamps for some files will change.

Ofc, the makefiles are configured to build the project into two different build directories .

Is there any way around this? Compilation is a very long and time-consuming process...

Edit:- This is a slightly more detailed explanation of the question... Say I have a header files Basic.h which is included in a number of other files. Basic.h is different between branch 1 and branch 2.

Now let's say I have compiled branch 1 into build_branch1 and branch 2 into build_branch2. Say I have branch 2 currently checked out. Now I checkout branch 1 and change File1.cpp and recompile. Ideally, since only File1.cpp has changed since I compiled it the last time, this is the only file that should be recompiled.

However, since Basic.h has it's timestamp changed due to the checkout, all files that are including Basic.h will get recompiled. I want to avoid this.

4
  • 2
    I thought git only changes the timestamps on files that have indeed changed. In that case, you have to rebuild anyway to get correct results. What behavior are you looking for?
    – sinelaw
    Commented Apr 27, 2012 at 18:50
  • 1
    See this related thread. Basically, doing something like this is not recommended.
    – mtvec
    Commented Apr 27, 2012 at 18:52
  • @sinelaw :- a.cpp includes Basic.h. I have two copies of a.o in branch1_build/a.o and branch2_build/a.o . Now when I switch branches from branch1 to branch2, it will recompile branch2_build/a.o even though it is up to date
    – owagh
    Commented Sep 22, 2016 at 16:38
  • Related: stackoverflow.com/questions/4913360/…
    – jozxyqk
    Commented May 24, 2018 at 23:29

5 Answers 5

17
+100

Git changes only the files that are updated between branches. But if your compiler does a full rebuild even if any single file was changed you can always clone and checkout your different branches into different directories. That's like:

/your-repo-name.branch1
/your-repo-name.branch2

This takes extra disk space but is much more convenient than switching divergent branches in a huge repo.

5
  • 7
    You can also use --shared to save most of that disk space: pick one of those clones as your "main" storage, and make the rest with git clone --shared referring back to the main-storage-clone. Just don't delete any branches in the "main" repo (or only delete ones that are not being shared) and you'll be safe there.
    – torek
    Commented Apr 27, 2012 at 19:24
  • 1
    Hey torek. Can you expand your answer about shared clones? Won't that create a shared repository that is group writable?
    – owagh
    Commented Apr 27, 2012 at 19:36
  • 1
    No, --shared only affects the new clone. In fact you don't even need --shared most of the time as git is (if not too ancient) smart enough to realize that it's a local clone and use hard links. (I imagine --shared is required on Windows though.)
    – torek
    Commented Apr 27, 2012 at 20:01
  • 1
    Well this is not exactly what I'd hoped to achieve but I guess this is the closest workable solution...
    – owagh
    Commented Apr 27, 2012 at 23:13
  • 3
    Also, you can use git worktree instead of full clones. That way you'll share the .git at least.
    – autra
    Commented Oct 24, 2018 at 7:06
12

Another partial answer: compiler cache.

When you switch back to the original branch and rebuild, although the dependencies say that numerous files dependent on Basic.h have to be rebuilt, the object files can be pulled from a compiler cache.

ccache ( http://ccache.samba.org/ ) still has to do some fairly expensive work (processing the pre-processed translation unit, since an entire translation unit is used a hash key) but it's a lot cheaper than compiling.

In some cases ccache can eliminate a compile in the face of a change that does not affect it, like some whitespace changes. For instance if you change a comment in a dependent file (header or source), that does not invalidate the cached object file.

So it can help even if you do a git pull and pick up a new change to Basic.h you haven't seen before.

0
4

If you know which files actually need compilation, and do those manually, GNU Make (at least, I don't know about other implementations) has a flag for you: -t, which basically runs over your Makefile and changes timestamps instead of running commands.

You'll still need to update the files that need updating before using this flag, or you'll end up with object files that are legitimately out-of-date but look updated. See the linked doc for details.

The -o option might also interest you, depending on how much changes when you switch branches.

3

If Basic.h actually differs between the branches, then the only fix is to break it up into smaller files with more fine-grained dependencies, so that less stuff is rebuilt when it changes.

But suppose that Basic.h is actually exactly the same between branches, but git is changing its timestamp. This is a false trigger, like doing touch Basic.h, which reveals a limitation of timestamp-based build systems. Ideally we want a build system to rebuild when contents change, not when timestamps change. Timestamps are used because they are a cheap substitute for detecting a content change. The superior method is for the build system to keep hashes of all files and detect actual modifications, without regard for the time stamp. This also fixes problems like "clock skew detected; your build may be incomplete".

You're not going to get this kind of build system out of Make; but you may be able to work some aspects of it with regard to certain files. For instance, you can write your Make rules such that your object files do not depend directly on Basic.h but on Basic.h.sha, which is a stamp file. Now, what is in Basic.h.sha? This file contains a SHA256 hash of Basic.h.

Every time you run Make, logic in the Makefile computes the hash over Basic.h and compares it to the hash stored in Basic.h.sha. If they differ, then Basic.h.sha is overwritten with the new hash. Thereby, its timestamp is bumped, triggering a rebuild.

BASIC_H_SHA := $(shell sha256sum Basic.h)
BASIC_H_SHA_OLD := $(shell cat Basic.h.sha)

ifneq ($(BASIC_H_SHA),$(BASIC_H_SHA_OLD))
$(info Basic.h has changed!)
DUMMY := $(shell echo "$(BASIC_H_SHA)" > Basic.h.sha)
endif

I have implemented logic along these lines to make modules dependent on changes to their respective CFLAGS. Basically, we can turn a change in anything into a touch of a timestamp file, which controls what is built.

3
  • 3
    I hope you do realize the ridiculousness of your answer. What I have given is the smallest subset that appropriately illustrates the problem at hand.
    – owagh
    Commented Apr 27, 2012 at 23:05
  • Obviously, you believe that although numerous files are recompiled because Basic.h changed, they do not actually have to be. This means that you believe that the files have some false dependencies on Basic.h (in addition to real ones). The only way to change that is to break up Basic.h, or else to introduce a completely different kind of build system which can manage fine grained dependencies such as "argument 3 of function foo changed; recompile everything that calls foo".
    – Kaz
    Commented Apr 27, 2012 at 23:25
  • 2
    The second part of your answer is a completely valid point about general programming techniques which I agree with but you have completely missed the whole point of my question and answered something I didn't even ask for. Thanks anyways...
    – owagh
    Commented Apr 30, 2012 at 14:32
2

There is no good way to preserve timestamps and not get into trouble in your build environment. Use git clone / git checkout branchB to create a second working directory for building and working with branch B. Don't have your Makefile build both, have each one build into its own build directory.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.