56

I have a repository on bitbucket that is using LFS. Since using it for some time, I've decided to move the repository back to a space under my control. The only reason I used LFS in the first place was to effectively double my repository size limit (as files in LFS go in a separate bucket) but now I'm moving it, I no longer need to do this.

I need a way to trawl through the entire git history, removing all traces of the work git LFS does (so all files are committed 'normally'). Once this is done, I intend to force push to the new repository.

I've done quite a bit of searching, and come across suggested solutions but I don't understand how to implement/run them because they are high-level.

How do I wave goodbye to git LFS?

2

5 Answers 5

53

Update current commit only

If you want to move off LFS, but are not so worried about fixing the entire git history, you can do the following;

git lfs uninstall
touch **/*
git commit -a

This will uninstall LFS support, touch every single file (so that git recognises that is has changed) then commit them all. If you like you could be more specific (ie, **/*.png for example). Note that using ** requires extended glob support enabled (shopt -s globstar on bash)

Update entire history

This worked for me - but it throws lots of errors (I think I'm getting an error for every commit that a file hasn't been added to LFS in) and takes a long time (roughly 2-3 seconds per commit).

git lfs uninstall
git filter-branch -f --prune-empty --tree-filter '
  git lfs checkout
  git lfs ls-files | cut -d " " -f 3 | xargs touch
  git rm -f .gitattributes
  git lfs ls-files | cut -d " " -f 3 | git add
' --tag-name-filter cat -- --all

It uninstalls git LFS support (theoretically preventing LFS from messing with the index) then for each commit it makes sure the LFS files are checked out properly, then touches them all (so git realises they have changed), removes the settings for LFS found in .gitattributes so that when cloning it doesn't keep trying to use LFS, then adds the real file to the index.

After you do the above, you will need to do a force push. Naturally, that'll throw anyone else working on your repo into a detached head state - so doing this during a code freeze is wise. Afterwards, it's probably easiest to get everyone to do a fresh clone.

2
  • I get this error on windows, even while using git for windows: cut' is not recognized as an internal or external command, operable program or batch file.
    – BikerDude
    Apr 15, 2022 at 16:49
  • 1
    @BikerDude yep, cut is a Linux command. You could use something like cygwin.
    – Shadow
    Apr 16, 2022 at 21:22
44

git lfs migrate export

From git lfs migrate help:

Export

The export mode migrates Git LFS pointer files present in the Git history out of Git LFS, converting them into their corresponding object files.

Example Workflow

  1. Verify you actually have LFS files with git lfs ls-files.
  2. Remove all filter=lfs lines from ALL the .gitattributes files in your repo. .gitattributes can live anywhere so make sure you find them all otherwise this can cause migration issues later.
  3. Commit any changes you made to .gitattributes.
  4. Make sure you have no changes with git status.
  5. Run the migration: git lfs migrate export --everything --include .
  6. Run git status to make sure you have no changes. If you left .gitattributes with filter=lfs you might incorrectly have changes now.
  7. Verify all the previously listed LFS files are no longer present with git lfs ls-files.
  8. Inspect files (e.g., open formerly LFS files to make sure they aren't corrupt) and run your build to make sure everything works.

Tips

  • Run on case sensitive file system, in case you have file system collisions (e.g. ./LICENSE and ./License) at some point.
  • Git rid of all your filter=lfs lines from ALL your .gitattributes.
  • You may also want to delete LFS remains in .git/hooks directory: pre-commit, post-commit, post-checkout, post-merge.
  • With $GIT_TRACE=1 there should be no sign of ...trace git-lfs: filepathfilter: accepting...
9
  • 2
    You may also want to delete LFS remains in .git/hooks directory: pre-commit, post-commit, post-checkout, post-merge. With $GIT_TRACE=1 there should be no sign of ...trace git-lfs: filepathfilter: accepting...
    – pizycki
    Nov 29, 2019 at 14:17
  • @pizycki Thanks. Added your comments to the answer. Dec 2, 2019 at 0:03
  • 2
    So this was awesome. Worked wonderfully whereas the simple instructions provided in the actual git lfs docs....not so much. Can you explain why the dot after --include? I always thought this had to be some expression but when I used essentially what was in my .gitattributes it didnt work by a long shot.
    – jkratz
    Apr 12, 2020 at 20:26
  • @jkratz It's a git <pathspec>. Note that the <pathspec> I used only works if you're at the top level directory in your git checkout. If you want to be able to run the command from anywhere like I intended you can use :/: instead. Run git ls-files <pathspec> to see how it works. git-scm.com/docs/gitglossary#Documentation/… Apr 12, 2020 at 20:52
  • 2
    What should you do if (e.g. on Github Desktop) it says your local is different from remote by all commits? Should I treat the exported repo as a completely new one? Feb 17, 2022 at 7:33
3

For particular file

  1. Remove file/pattern from .gitattributes
  2. Go to file dir and run touch myfile.bin
  3. Commit and push changes
1

Based on the answer of Shadow I modified entire history update a little. For the ones, who do not want to keep LFS files (which in my case I did not, because they would've been huge).

Use:

GIT_LFS_SKIP_SMUDGE=1 git filter-branch -f --prune-empty --tree-filter '
  git lfs checkout
  git lfs ls-files | cut -d " " -f 3 | xargs rm -f
  git rm -f --ignore-unmatch .gitattributes
' --tag-name-filter cat -- --all

Also it does not fail when no .gitattributes is present in a commit, which again, in my case was not true for all the commits.

Also the repo did not have the original LFS files stored in remote, hence the usage of GIT_LFS_SKIP_SMUDGE=1

This way the repo does not include any LFS references and files, which makes cloning the repo faster and lighter - which was my goal. Large files were used for tests anyway, but since code has improved a lot, running those old tests is irrelevant.

0

I wrote a script that completely removes lfs from a git repo. WARNING - it rewrites your history.

#!/bin/zsh

git lfs install
git lfs fetch --all
git lfs checkout

lfs_dirs=()

echo "finding lfs files"
for lfs_file in $(git lfs ls-files | sed -r 's/^.{13}//'); do lfs_dirs+=($(dirname "$lfs_file")); done;

echo "found all lfs files"

unique_dirs=()
for unique_dir in ${(u)lfs_dirs}; do unique_dirs+=("$unique_dir/*"); done;

count=${#unique_dirs[@]}

if (($count == 0))
then
echo "No lfs dirs"
exit 0
fi

echo "There are $count unique lfs dirs"

printf -v migrate_paths ',%s' "${unique_dirs[@]}"
migrate_paths=${migrate_paths:1}

echo "performing: git lfs migrate export --include='$migrate_paths' --yes --verbose --everything"
git lfs migrate export --include="$migrate_paths" --yes --verbose --everything
git lfs uninstall
1
  • Thanks for the response. As per the stackoverflow guidelines, link only answers such as this are discouraged, as links can break. Would you consider copying your script into your answer?
    – Shadow
    Mar 20 at 12:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.