The backend of Git itself is text-based, which is not suited for SQL queries.
That might change someday, as illustrated with Git 2.44 (Q1 2024), which introduces a new extension "refstorage
" so that we can mark a repository that uses a non-default ref backend, like reftable.
See commit 1b22340, commit 5ed860f, commit 48fa45f, commit 3c4a531, commit 58aaf59, commit aa19619, commit d7497a4, commit 58be32f, commit 173761e, commit 0fcc285, commit 465a22b, commit bb0372c (29 Dec 2023) by Patrick Steinhardt (pks-t
).
See commit 7a75e13 (20 Dec 2023) by Junio C Hamano (gitster
).
(Merged by Junio C Hamano -- gitster
-- in commit 32c6fc3, 16 Jan 2024)
builtin/clone
: introduce --ref-format=
value flag
Signed-off-by: Patrick Steinhardt
Introduce a new --ref-format
value flag for git-clone
(1) that allows the user to specify the ref format that is to be used for a newly initialized repository.
git clone
now includes in its man page:
--ref-format=<ref-format>
Specify the given ref storage format for the repository. The valid values are:
files
for loose files with packed-refs. This is the default.
Meanwhile, (2023/2024), consider the project dolthub/dolt
Dolt is a SQL database that you can fork, clone, branch, merge, push and pull
just like a Git repository.
Connect to Dolt just like any MySQL database to read or modify schema
and data. Version control functionality is exposed in SQL via system
tables, functions, and procedures.
Or, use the Git-like command line interface to import CSV files,
commit your changes, push them to a remote, or merge your teammate's
changes. All the commands you know for Git work exactly the same for
Dolt.
Git versions files. Dolt versions tables. It's like Git and MySQL had
a baby.
But, with that new backend, "git init
"(man) in an already created directory, when the user configuration has includeif.onbranch
, started to fail recently, which has been corrected with Git 2.46 (Q3 2024), batch 10.
See commit 407997c (22 May 2024) by Patrick Steinhardt (pks-t
).
(Merged by Junio C Hamano -- gitster
-- in commit 5c7c063, 03 Jun 2024)
setup
: fix bug with "includeIf.onbranch
" when initializing dir
Reported-by: Heghedus Razvan
Signed-off-by: Patrick Steinhardt
Tested-by: Heghedus Razvan
It was reported that git init
can fail when initializing an existing directory in case the config contains an "includeIf.onbranch:
" condition:
$ mkdir repo
$ git -c includeIf.onbranch:main.path=nonexistent init repo
BUG: refs.c:2056: reference backend is unknown
The same error can also be triggered when re-initializing an already existing repository.
The bug has been introduced in 173761e ("setup
: start tracking ref storage format", 2023-12-29, Git v2.44.0-rc0 -- merge listed in batch #8), which wired up the ref storage format.
The root cause is in init_db()
, which tries to read the config before we have initialized the_repository
and most importantly its ref storage format.
We eventually end up calling include_by_branch()
and execute refs_resolve_ref_unsafe()
, but because we have not initialized the ref storage format yet this will trigger the above bug.
Interestingly, include_by_branch()
has a mechanism that will only cause us to resolve the ref when the_repository->gitdir
is set.
This is also the reason why this only happens when we initialize an already existing directory or repository: gitdir
is set in those cases, but not when creating a new directory.
Rearrange the code such that we set up the repository format before reading the config.
This fixes the bug and ensures that "onbranch:
" conditions can trigger.
With Git 2.46 (Q3 2024), batch 14, a new command has been added to migrate a repository that uses the files backend for its ref storage to use the reftable backend, with limitations.
See commit 25a0023, commit 6d6a3a9, commit 64a6dd8, commit 1339cb3, commit b5d7db9, commit b3e098d, commit 120b671, commit 66275a6, commit fbd1a69, commit 6e1683a, commit 318efb9, commit a83f7f5 (06 Jun 2024) by Patrick Steinhardt (pks-t
).
See commit 0ff6d23, commit e55f364 (23 May 2024) by Junio C Hamano (gitster
).
(Merged by Junio C Hamano -- gitster
-- in commit 40a163f, 17 Jun 2024)
builtin/refs
: new command to migrate ref storage formats
Signed-off-by: Patrick Steinhardt
Introduce a new command that allows the user to migrate a repository between ref storage formats.
This new command is implemented as part of a new git-refs(1)
executable.
This is due to two reasons:
- There is no good place to put the migration logic in existing commands.
git-maintenance(1)
felt unwieldy, and git-pack-refs(1)
is not the correct place to put it, either.
- I had it in my mind to create a new low-level command for accessing refs for quite a while already.
git-refs(1)
is that command and can over time grow more functionality relating to refs.
This should help discoverability by consolidating low-level access to refs into a single executable.
As mentioned in the preceding commit that introduces the ref storage format migration logic, the new git refs migrate
(man) command still has a bunch of restrictions.
These restrictions are documented accordingly.
git refs
now includes in its man page:
git-refs(1)
git-refs
- Low-level access to refs
git refs migrate --ref-format=<format> [--dry-run]
This command provides low-level access to refs.
migrate
Migrate ref store between different formats.
The following options are specific to 'git refs migrate
':
--ref-format=<format>
The ref format to migrate the ref store to. Can be one of:
files
for loose files with packed-refs. This is the default.
reftable
for the reftable format. This format is experimental and its internals are subject to change.
--dry-run
Perform the migration, but do not modify the repository.
The migrated refs will be written into a separate directory that can be inspected separately.
The name of the directory will be reported on stdout.
This can be used to double check that the migration works as expected before performing the actual migration.
KNOWN LIMITATIONS
The ref format migration has several known limitations in its current form:
It is not possible to migrate repositories that have reflogs.
It is not possible to migrate repositories that have worktrees.
There is no way to block concurrent writes to the repository during an
ongoing migration. Concurrent writes can lead to an inconsistent migrated
state. Users are expected to block writes on a higher level. If your
repository is registered for scheduled maintenance, it is recommended to
unregister it first with git-maintenance
.
These limitations may eventually be lifted.