6

I put together tarball'd releases of software that include the output from several different projects. These tarballs themselves are considered a release. The released tarball includes a BOM (bill of materials) that itemizes all the projects in it and their associated SHA1 (git) signatures.

To make recreating these tarballs easier, I put together a make system that parses the BOM line-by-line cloning the repository into a subdirectory, checking out the specified version then performing the build. Say a line in my BOM has:

prog-a b5286f27d65ef20eb4508f76de5a1c57d8b21d85 git+ssh://git-user@localhost/~/prog-a

the repository, if not already cloned, would be placed in repos/prog-a, then a checkout is done (cd repos/prog-a; git checkout b5286f27d6) and finally a make (make -C repos/prog-a).

I've not figured out how to let gnu make decide if the checked out version of the code has already built the binary I need. Currently every single sub-project is forced to checkout and rebuild.

How can I present a git repo's SHA1 to GNU make so that it can decide if the project is out of date and needs to be updated (by performing a git checkout)?

[EDIT] This is my pattern file:

REPO_DIR=materials
BOM=$(shell sed -r 's/([^ ]+).+/\1/' bom)
BOM_DIR=$(shell sed -r 's_([^ ]+).+_$(REPO_DIR)/\1_' bom)
BOM_BLD=$(shell sed -r 's_([^ ]+).+_$(REPO_DIR)/\1/\1_' bom)

.PHONY: clean dist-clean

all: $(BOM)

clean:
    @rm $(BOM) $(BOM_BLD) -rf

dist-clean: clean
    @rm $(REPO_DIR)

.SECONDEXPANSION:

$(BOM): % : $(REPO_DIR)/$$*/$$*
    @echo " CP $< $@"
    @cp $< $@

$(BOM_BLD): % :  $$(*D)
    @echo " GIT CHECKOUT"
    @cd $<; git checkout -q $(shell sed -rn '/$(shell echo $@ | sed -r 's_.+/__')/ s/.+ (.+) .+ .+ .+/\1/p' bom)
    @echo " MAKE $@"
    @make -w -C $< $(@F)

$(BOM_DIR): | materials
    @echo " GIT CLONE $@"
    @cd $(REPO_DIR); git clone $(shell sed -rn '/$(shell echo $@ | sed -r 's_.+/__')/ s/.+ (.+) .+ .+/\1/p' bom)

materials:
    @echo " MKDIR $@"
    @mkdir $@
8
  • If the checkout only replaces files that needs replacing, you could just do the checkout unconditionally. Are you saying it replaces more than it needs to?
    – ikegami
    Commented Sep 1, 2011 at 1:51
  • No, the checkout works just fine, as do the sub-project makes. What I'd like to have is the top level make see which checked out version a sub-project is, and determine if it is the one that's required/specified by the BOM line.
    – Jamie
    Commented Sep 1, 2011 at 1:55
  • 1
    Just do the checkout. make will determine whether anything needs to be done or not based on file timestamps.
    – ikegami
    Commented Sep 1, 2011 at 2:03
  • I see what you're saying. I wasn't being clear. What I was trying to avoid was running the make for the sub-projects altogether, which as you pointed out wouldn't take long if I'm checking out the same version of sub-project.
    – Jamie
    Commented Sep 1, 2011 at 2:21
  • To simplify things you could consider tags per 'subfolder' (prog-a, prog-b). Referring to the commits will become much easier. You could also resort to git submodules (sharing a shared repo) and profit from git's submodule handling
    – sehe
    Commented Sep 1, 2011 at 12:39

2 Answers 2

1

This would be my pattern:

TARGETS=prog-a prog-b prog-c

all: $(TARGETS)

prog-a: SHA1=123fa
prog-b: SHA1=234ab
prog-c: SHA1=345bc

$(TARGETS):
    make -C "$@" -e PARAM=$(SHA1)

In your subdir makefile, imagine something like this:

all:
    git checkout $(PARAM) -- ./
    # ... other build rules

assuming makefiles in the subdirs; you can of course do whatever you want in the make rule.

For even more dynamic make scripts, at least have a look at .SECONDEXPANSION, .PHONY, .PRECIOUS

2
  • update: showed example of where the git checkout would fit in; you could of course lift all/some of the logic into the toplevel makefile.
    – sehe
    Commented Sep 1, 2011 at 8:56
  • Thanks for taking the time to respond, I'm already doing something like that (see my forthcoming edit). My question is still 'How can I have make make the decision to proceed with pattern rule to do the checkout or not?'
    – Jamie
    Commented Sep 1, 2011 at 13:23
0

I ended up doing an unconditional checkout and just took the hit on time if it actually had to manipulate things in the working directory of the bom item.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.