We tried to speed up the CI build of one of our software projects at work. Somebody committed some huge (by git's standards) binaries early in the project's life. Rewriting git's history just to get rid of them seems like too much trouble, so we figured doing a shallow clone that avoided those big early commits would be good enough.
I did some experiments with --depth
parameter for clone and encountered some weird behavior. This is what help for git clone says about it:
--depth <depth>
Create a shallow clone with a history truncated to the specified number of commits. Implies
--single-branch unless --no-single-branch is given to fetch the histories near the tips of all
branches. If you want to clone submodules shallowly, also pass --shallow-submodules.
This would indicate that <depth>
will equal the number of commits that will be fetched during the clone, but it's not the case. This is what I got when I tried different values for depth:
| depth | commit count linux repo | commit count git repo |
|---------|-------------------------|-----------------------|
| 1 | 1 | 1 |
| 5 | 15 | 13 |
| 10 | 80 | 46 |
| 100 | 93133 | 39552 |
| 1000 | 788718 | 53880 |
For cloning I used this command git clone --depth 10 https://github.com/torvalds/linux.git
, git clone --depth 100 https://github.com/git/git.git
, and for counting the commits I used this git log --oneline | wc -l
. (At work I observed the same thing with a GitLab server, so it can't be an artifact of how GitHub works.)
Does anybody know what is going on? How does the value for depth correspond to the actual amount of data downloaded? Do I understand the documentation wrongly, or is there a bug?
EDIT: I added results for a second repo
git clone --filter=blob:none <url>
this will get you everything (i.e. you can dogit fetch
on it later) without those huge files.--filter
was introduced in git 2.27.0.