1

We have a large project consisting of the following:

  • A: C++ source code / libraries
  • B: Java and Python wrapping of the C++ libraries, using SWIG
  • C: a GUI written in Java, that depends on the Java API/wrapping.

People use the project in all the possible ways:

  • C++ projects using the C++ API
  • Java projects using the Java API
  • Python scripting
  • MATLAB scripting (using the Java API)
  • through the Java GUI

Currently, A, B and C are all in a single Subversion repository. We're moving to git/GitHub, so we have an opportunity to reorganize. We are thinking of splitting A, B, and C into their own repositories. This raises a few questions for us:

  1. Does it make sense to split off the Java and Python SWIG wrapping (that is, the interface (*.i) files) into a separate repository?
  2. Currently, SWIG-generated .java files are output in the source tree of the GUI and are tracked in SVN. Since we don't want to track machine-generated files in our git repositories, what is the best way of maintaining the dependency of the GUI on the .java/.jar files generated by SWIG? Consider this: if a new developer wants to build the Java GUI, they shouldn't need to build A and B from scratch; they should be able to get a copy of C from which they can immediately build the GUI.
  3. Versioning: When everything is in one repository, A, B and C are necessarily consistent with each other. When we have different repositories, the GUI needs to work with a known version of the wrapping, and the wrapping needs to work with a known version of the C++ API. What is the best way to manage this?

We have thought deeply about each of these questions, but want to hear how the community would approach these issues. Perhaps git submodule/subtree is part of the solution to some of these? We haven't used either of these, and it seems submodules cause people some headache. Does anybody have stories of success with either of these?

1 Answer 1

0

OK, I looked in a similar problem like you (multiple interacting projects) and I tried the three possibilities subtree, submodules and a single plain repository with multiple folders containing the individual parts. If there are more/better solutions I am not aware of them.

In short I went for a single repository but this might not be the best solution in your case, that depends...

  • The benefit of submodules is that it allows easy management as every part is itself a repo. Thus individual parties can work only on their repo and the other parts can be added from predefined binary releases/... (however you like). You have to add an addtional repo that concatenates the individual repos together.
    This is both the advantage and disadvantage: Each commit in this repo defines a running configuration. Ideally your developers will have to make each commit twice one for the "working repo" (A through C) and one for the configuration repo.
    So this method might be well suited if you intent you parts A-C to be mostly independent and not changing too often (that is only for new releases).
  • I have to confess that I did not like the subtree method personally. For me (personally) the syntax seems clumsy and the benefit is not too large.
    The benefit is that remote modifications are easily fetched and inserted but you loose the remote history. Thus you should avoid to interfere with the remote development.
    This is the downside: If you intend to do modifications on the parts you have always to worry about the history. You can of course just develop in a git remote and for testing/merging/integrating change to the master branch. This is ok for mainly reading remote repos (if I am developing only on A but need B and C) but not for regular modifications (in the example for A).
  • The last possibility is one plain repo with folders for each part. The benefit is that no adminstration to keep the parts in sync is directly needed. However you will not be able to guarantee that each commit will be a running commit. Also you developers will have to do the administration by hand.

You see that the choice depends on how close the individual parts A-C are interconnected. Here I can only guess:
If you are in an earlier stage of development where modifications throughout the whole source tree are common one big repo is better handleable than a splitted version. If your interfaces are mostly constant the splitting allows smaller repos and a more strict separation of things.
The SWIG code and the C++ code seems quite close. Thus splitting those two seems less practical than splitting the GUI from the rest (for my guess).

For you other question "How to handle new developers/(un)tracking machine-generated code?":
How many commits are made/required (only releases or each individual commit)? If only releases are of interest you could go with binary packages. If you intent to share each single commit, you would have to provide many different binary versions. Here I would suggest let them compile the whole tree once consuming a few minutes and from there on rebuilding is just a short make that should not take too long. This could even automatized using hooks.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.