69

I have a repository on bitbucket that is using LFS. Since using it for some time, I've decided to move the repository back to a space under my control. The only reason I used LFS in the first place was to effectively double my repository size limit (as files in LFS go in a separate bucket) but now I'm moving it, I no longer need to do this.

I need a way to trawl through the entire git history, removing all traces of the work git LFS does (so all files are committed 'normally'). Once this is done, I intend to force push to the new repository.

I've done quite a bit of searching, and come across suggested solutions but I don't understand how to implement/run them because they are high-level.

How do I wave goodbye to git LFS?

3

7 Answers 7

68

Update current commit only

If you want to move off LFS, but are not so worried about fixing the entire git history, you can do the following;

git lfs uninstall
touch **/*
git commit -a

This will uninstall LFS support, touch every single file (so that git recognises that is has changed) then commit them all. If you like you could be more specific (ie, **/*.png for example). Note that using ** requires extended glob support enabled (shopt -s globstar on bash)

Update entire history

This worked for me - but it throws lots of errors (I think I'm getting an error for every commit that a file hasn't been added to LFS in) and takes a long time (roughly 2-3 seconds per commit).

git lfs uninstall
git filter-branch -f --prune-empty --tree-filter '
  git lfs checkout
  git lfs ls-files | cut -d " " -f 3 | xargs touch
  git rm -f .gitattributes
  git lfs ls-files | cut -d " " -f 3 | git add
' --tag-name-filter cat -- --all

It uninstalls git LFS support (theoretically preventing LFS from messing with the index) then for each commit it makes sure the LFS files are checked out properly, then touches them all (so git realises they have changed), removes the settings for LFS found in .gitattributes so that when cloning it doesn't keep trying to use LFS, then adds the real file to the index.

After you do the above, you will need to do a force push. Naturally, that'll throw anyone else working on your repo into a detached head state - so doing this during a code freeze is wise. Afterwards, it's probably easiest to get everyone to do a fresh clone.

6
  • I get this error on windows, even while using git for windows: cut' is not recognized as an internal or external command, operable program or batch file.
    – BikerDude
    Commented Apr 15, 2022 at 16:49
  • 1
    @BikerDude yep, cut is a Linux command. You could use something like cygwin.
    – Shadow
    Commented Apr 16, 2022 at 21:22
  • 1
    I suggest to avoid git lfs uninstall until well after Git LFS is not used at your organization because if some coworkers keep using Git LFS (or merge commits that are not descendant from this Git LFS removal commit), you will end up wit Git LFS pointer files in your worktree instead of the files. Also the "current commit only" answer does not update the .gitattributes which is likely to cause trouble in the future. See my answer for an alternative solution. Commented Aug 22 at 14:32
  • 1
    @RomainValeri the files have changed because the content has been restored to them. So the act of touching everything indeed won't effect text files, or any files that weren't previously lfs'd.
    – Shadow
    Commented Sep 11 at 3:17
  • 1
    @Shadow Thanks, got it. I disregarded the lfs uninstall context. Commented Sep 11 at 8:11
50

git lfs migrate export

From git lfs migrate help:

Export

The export mode migrates Git LFS pointer files present in the Git history out of Git LFS, converting them into their corresponding object files.

Example Workflow

  1. Verify you actually have LFS files with git lfs ls-files.
  2. Remove all filter=lfs lines from ALL the .gitattributes files in your repo. .gitattributes can live anywhere so make sure you find them all otherwise this can cause migration issues later.
  3. Commit any changes you made to .gitattributes.
  4. Make sure you have no changes with git status.
  5. Run the migration: git lfs migrate export --everything --include .
  6. Run git status to make sure you have no changes. If you left .gitattributes with filter=lfs you might incorrectly have changes now.
  7. Verify all the previously listed LFS files are no longer present with git lfs ls-files.
  8. Inspect files (e.g., open formerly LFS files to make sure they aren't corrupt) and run your build to make sure everything works.

Tips

  • Run on case sensitive file system, in case you have file system collisions (e.g. ./LICENSE and ./License) at some point.
  • Git rid of all your filter=lfs lines from ALL your .gitattributes.
  • You may also want to delete LFS remains in .git/hooks directory: pre-commit, post-commit, post-checkout, post-merge.
  • With $GIT_TRACE=1 there should be no sign of ...trace git-lfs: filepathfilter: accepting...
11
  • 3
    You may also want to delete LFS remains in .git/hooks directory: pre-commit, post-commit, post-checkout, post-merge. With $GIT_TRACE=1 there should be no sign of ...trace git-lfs: filepathfilter: accepting...
    – pizycki
    Commented Nov 29, 2019 at 14:17
  • @pizycki Thanks. Added your comments to the answer. Commented Dec 2, 2019 at 0:03
  • 2
    So this was awesome. Worked wonderfully whereas the simple instructions provided in the actual git lfs docs....not so much. Can you explain why the dot after --include? I always thought this had to be some expression but when I used essentially what was in my .gitattributes it didnt work by a long shot.
    – jkratz
    Commented Apr 12, 2020 at 20:26
  • 2
    What should you do if (e.g. on Github Desktop) it says your local is different from remote by all commits? Should I treat the exported repo as a completely new one? Commented Feb 17, 2022 at 7:33
  • 3
    After doing this and pushing, I got hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Integrate the remote changes (e.g. hint: 'git pull ...') before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. when I then tried to pull I got fatal: refusing to merge unrelated histories. Not sure what I did wrong.
    – Pro Q
    Commented Nov 21, 2022 at 3:28
4

I would like to add a note to @Doug Richardson 's answer.

Not sure why but git lfs migrate export --everything --include . did not work for me.

Then, I found that the answer by @simon-the-shark in this git-lfs issue post works. Here is the git command that he suggested.

git lfs migrate export --include="*" --everything

After git lfs migrate export, I removed all git hooks, following @wotanii's answer here. I think it restores my git push behaviour to default. If I didn't do this, git push will try to push using lfs.

rm -f .git/hooks/*

Background: I am reversing the lfs setting of my local repo because the git-lfs authentication in my company's repo is not working, so when I do git push it always fails. Btw, I used GIT_TRACE=1 git push to find out the root cause of this problem.

3

For particular file

  1. Remove file/pattern from .gitattributes
  2. Go to file dir and run touch myfile.bin
  3. Commit and push changes
1

Based on the answer of Shadow I modified entire history update a little. For the ones, who do not want to keep LFS files (which in my case I did not, because they would've been huge).

Use:

GIT_LFS_SKIP_SMUDGE=1 git filter-branch -f --prune-empty --tree-filter '
  git lfs checkout
  git lfs ls-files | cut -d " " -f 3 | xargs rm -f
  git rm -f --ignore-unmatch .gitattributes
' --tag-name-filter cat -- --all

Also it does not fail when no .gitattributes is present in a commit, which again, in my case was not true for all the commits.

Also the repo did not have the original LFS files stored in remote, hence the usage of GIT_LFS_SKIP_SMUDGE=1

This way the repo does not include any LFS references and files, which makes cloning the repo faster and lighter - which was my goal. Large files were used for tests anyway, but since code has improved a lot, running those old tests is irrelevant.

1

I wrote a script that completely removes lfs from a git repo. WARNING - it rewrites your history.

#!/bin/zsh

git lfs install
git lfs fetch --all
git lfs checkout

lfs_dirs=()

echo "finding lfs files"
for lfs_file in $(git lfs ls-files | sed -r 's/^.{13}//'); do lfs_dirs+=($(dirname "$lfs_file")); done;

echo "found all lfs files"

unique_dirs=()
for unique_dir in ${(u)lfs_dirs}; do unique_dirs+=("$unique_dir/*"); done;

count=${#unique_dirs[@]}

if (($count == 0))
then
echo "No lfs dirs"
exit 0
fi

echo "There are $count unique lfs dirs"

printf -v migrate_paths ',%s' "${unique_dirs[@]}"
migrate_paths=${migrate_paths:1}

echo "performing: git lfs migrate export --include='$migrate_paths' --yes --verbose --everything"
git lfs migrate export --include="$migrate_paths" --yes --verbose --everything
git lfs uninstall
1
  • Thanks for the response. As per the stackoverflow guidelines, link only answers such as this are discouraged, as links can break. Would you consider copying your script into your answer?
    – Shadow
    Commented Mar 20, 2023 at 12:41
1

Here is the method that worked for me in 2024, to stop using Git LFS entirely. Inspired from this answer but cleaner (minimal change) on some Windows setups.

This method does not rewrite the history: the removal is done in a new commit.

Be careful:

  • if multiple people work in the same repository and have created branches before this "Git LFS removal" commit and have added files in LFS in those branches, they will also have to do this removal operation in their branches. If the team does not want to use Git LFS it seems a good idea to check it in the CI (git lfs ls-files --name-only must return nothing)
  • I do not recommend running git-lfs uninstall until the CI checks that Git LFS is not used, in case some other team member merges files that are in Git LFS (via a branch created before the "Git LFS removal" commit).
  • if you run on Windows then it is likely you use the Git configuration core.autocrlf=true (this is the default). This can make this operation more tricky if some text files actually have CRLF EOL in the commits (instead of having CRLF EOL only in the worktree). This is why this tutorial works from a new temporary clone with setting core.autocrlf=false (else on Windows some text files could be modified on end of lines which would pollute your git blames). We suggest working from a new temporary clone rather than messing with your Git configuration.

Another way of checking that a file is not in Git LFS is to use GitHub/GitLab/etc. web interface to browse the files in the commit. Files in Git LFS have a tag on them:

file with Git LFS tag in GitLab.

To remove only some files from Git LFS only remove lines in .gitattributes files related to those files.

Windows solution

(i.e. when your preexisting clone uses core.autocrlf=true)

git clone --config core.autocrlf=false my-project-url temporary-project-clone # motivated by explanation above

cd temporary-project-clone

git switch development-branch (if default branch is not the development branch)

git switch -c remove-git-lfs # unless you push directly to the development branch, which I do not recommend

# edit .gitattributes files (there can be multiple ones) to remove all lines related to Git LFS, example of such line:
# *.gltf filter=lfs diff=lfs merge=lfs -text

git add --renormalize . # in the root directory of the clone

git commit -m "Stop using Git LFS"

# check no more files are managed by Git LFS:
git lfs ls-files --name-only # should display nothing

git push

# unless you pushed directly to the development branch, follow your usual review & CI process to merge into the development branch

cd ..

rm -rf temporary-project-clone # if you are on Windows you do not want to work with config.coreautocrlf=false if you/your team usually use true

Linux / macOS / Unix solution

(i.e. when your preexisting clone does not use core.autocrlf=true)

Do start from a clean and up to date state (or unclean but only due to Git LFS related problems ...)

git switch -c remove-git-lfs # unless you push directly to the development branch, which I do not recommend

# edit .gitattributes files (there can be multiple ones) to remove all lines related to Git LFS, example of such line:
# *.gltf filter=lfs diff=lfs merge=lfs -text

git add --renormalize . # in the root directory of the clone

git commit -m "Stop using Git LFS"

# check no more files are managed by Git LFS:
git lfs ls-files --name-only # should display nothing

git push

# unless you pushed directly to the development branch, follow your usual review & CI process to merge into the development branch

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.