256

How to clean repo, if staged files marked as modified?

After

git reset --hard

I get

Encountered 7 file(s) that should have been pointers, but weren't:

Running git clean -fdx (which deletes everything, including untracked directories and anything in .gitignore) doesn't help, either.

5
  • 1
    That error message sounds like something you would get from git-lfs. I don't actually use git-lfs so I'm not sure about this (nor what to do about it), but if so, maybe the git-lfs tag would be good.
    – torek
    Oct 12, 2017 at 16:52
  • yes, lfs is used
    – Kate Zz
    Oct 13, 2017 at 14:41
  • @KateZz did you ever find an answer to this? We use git-lfs as well and I just checked out a branch and got that that error. Dec 21, 2017 at 15:55
  • 14
    It's easy to run into this situation if you add filter=lfs diff=lfs merge=lfs entries to your .gitattributes that match already commited files. If you want to make sure, they are converted at the same time, use git rm --cached . and git add -A to switch them to LFS pointers. (Of course assuming you are on an otherwise clean working directory.) If you forget to convert them, the problem may show up much later. I am not sure when exactly - probably when they get touched somehow. Jul 8, 2020 at 9:03
  • Doing a git lfs pull in the repo before committing resolved this issue for me
    – CPayne
    Aug 16, 2022 at 23:42

23 Answers 23

430

Like Travis Heeter mentioned in his answer, Try the following command sequence:

git lfs uninstall
git reset --hard
git lfs install
git lfs pull

In case if this is not working (because this was not working for me), the following hack may work:

git rm --cached -r .
git reset --hard
git rm .gitattributes
git reset .
git checkout .

This worked for me!

12
  • 3
    After battling with this across different branches and doing things like making temp branches and committing the problem file to those instead, I still found I had one branch left with a Unity asset file stuck behind. This method worked brilliantly.
    – Vixxd
    Mar 15, 2019 at 22:38
  • 3
    I needed to do a merge with another branch, but could not get rid of those non-pointer files. Driving me crazy... So, I did a git lfs uninstall, git reset hard <commit>, git merge <branch>, git lfs install and git lfs pull. Finally :). Thanks.
    – Spiralis
    May 6, 2019 at 12:13
  • 3
    Ops, my previous comment should have said git reset --hard... No way to edit in Stackoverflow after a while, sigh...
    – Spiralis
    May 6, 2019 at 22:02
  • 2
    Worked for me BUT on windows, had to use PowerShell rather than git bash
    – Avi Turner
    Nov 28, 2020 at 17:28
  • 2
    This doesn't work for me. git reset --hard, the 2nd step on either path, fails with the same message as the questioner so I can't get past that step
    – gman
    Jan 29, 2021 at 8:32
157

I had this exact error with some files stored with git-LFS and solved it the same way I've solved a linending induced borked index .

Clear the cache and do a hard reset:

git rm --cached -r .
git reset --hard

This was significantly faster than a fresh clone for me due to the huge git-LFS files in my repo.

5
  • 4
    This didn't work for me, but the second code block of AnsarAdimu's answer did. Didn't try the first code block of their answer because uninstalling git lfs seemed a bit much.
    – mic
    May 8, 2020 at 18:34
  • 1
    This worked for me. I think the error was originally caused by files being checked in by a user NOT using Git LFS, then pulled down to a user who DID have Git LFS, which then didn't know how to deal with it. So those non-Git-LFS-managed files need to be removed from the repository. Oct 30, 2021 at 0:00
  • 2
    This didn't work for me, either. Nov 1, 2021 at 21:06
  • This worked for me. My issue was I changed the .gitattributes file on a feature branch to include all the files that should have been in git LFS and then when I merged to main it caused this issue. Dec 31, 2021 at 4:06
  • Is git rm --cached -r . is redundant with git reset --hard?
    – Schwern
    Feb 3, 2022 at 17:21
118

Since git lfs 2.5.0, there is a new command available that makes this easier (docs):

git lfs migrate import --no-rewrite "broken file.jpg" "another broken file.png" ...

This "migrates" files to git lfs which should be in lfs as per .gitattributes, but aren't at the moment (which is the reason for your error message).

--no-rewrite prevents git from applying this to older commits, it creates a single new commit instead.

Use -m "commitmessage" to set a commitmessage for that commit.

8
  • 2
    This solution worked for me. I originally found it from this blog post: tech-notes.maxmasnick.com/… Nov 26, 2019 at 14:16
  • 1
    Does no-rewrite mean there will still possibly be multiple binary versions in your git history instead of just pointers?
    – pooya13
    Apr 7, 2021 at 19:19
  • 1
    yes. old commits won't be changed, so whatever is contained in there stays in the repo. if you want to wipe the file from all commits, you'll need to rewrite the history (omitting --no-rewrite would do that I assume)
    – karyon
    Apr 8, 2021 at 10:05
  • 5
    This should be the top-ranked answer. Works! Jul 22, 2021 at 12:37
  • 2
    Agree with @BrianCraig. Should be top answer as of 2021
    – user585968
    Sep 9, 2021 at 22:08
73

The problem comes from the mismatch beetween the filetypes marked as to be tracked by git LFS in the .gitattributes and some matching files already under conventional non-LFS version control.

So the simplest workaround here is to just remove the .gitattributes file for a moment:

git rm .gitattributes
git reset .
git checkout .

Afterwards you can checkout any other branch.

One more advice: When adding a new filetype to git LFS, prefer not to do this by hand by modifying the .gitattributes but e.g. by running:

git lfs track PATTERN

where PATTERN is the pattern for matching files, e.g *.so

This way all already non-LFS versioned files matching the new tracking-pattern will be marked dirty and can be simlpy added, i.e. converted to git LFS (file pointers).

4
  • 2
    I can confirm this almost always works with Git 2.18.0, 2.26.2 and 2.27.0. If it doesn't work in first try, delete the offending files and run above commands again. Jul 9, 2020 at 7:43
  • 5
    "just remove the .gitattributes file for a moment" Could you explain how those three command do that? It seems to delete .gitattributes and stage the delete, then undo the stage, then undo the delete. The net result seems to be nothing. Is there some git-lfs magic?
    – Schwern
    Feb 3, 2022 at 17:24
  • Finally, this approach worked for me...
    – robert
    May 10 at 8:34
  • Had to do a slight variant of @keremispirli 's suggestion. Delete the file, execute the first two commands in this answer, then do a git pull, then run the final command. This seems to have cleared things up everywhere.
    – ouflak
    Sep 7 at 12:49
31

This can happen when you do a checkout that contains files which should have been been tracked by LFS as specified in .gitattributes but somehow they've been committed directly instead. Most likely you have another program managing your repository such as a git GUI or IDE.

This can be frustrating because these files appear out of nowhere and prevent you from making checkouts. As soon as you stash your changes they return! If you get stuck in this situation, a quick fix is to commit these changes on a temporary branch so that you can checkout again.

To genuinely fix this problem, make sure you've committed the files as LFS pointers. This should be as simple as using git add. Check your work using git lfs status before committing. git lfs ls-files will show what files LFS is managing.

git lfs status is misleading since it reads Git LFS objects to be committed when it really lists all changes. Files that you expect to be tracked by LFS should read something like (LFS: c9e4f4a) or (Git: c9e4f4a -> LFS: c9e4f4a) and not (Git: c9e4f4a).

By way of example, I found this to be a problem when adding image assets through Xcode 9.2 where I added "CalendarChecked.png" which it automatically added:

$ git status
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    new file:   Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png

$ git lfs status

Git LFS objects to be committed:

    Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png (Git: c9e4f4a)

Git LFS objects not staged for commit:

    Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png (File: c9e4f4a)

$ git add Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png`
$ git lfs status

Git LFS objects to be committed:

    Empty/Empty/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png (LFS: c9e4f4a)

Git LFS objects not staged for commit:

$
30

Neither of these solutions worked for me, but I pieced together a few sources to finally get all this fixed.

  1. Push any changes you don't want to lose

    If you can... If not, or if you don't care about your changes, press on.

  2. Stop Everything

    SourceTree, any servers, file explorers and browsers. Sometimes this stuff won't work if it's being used somewhere else. When in doubt, stop it - with this it's better to overkill.

    Also, go into Task Manager, force quit any bash.exe processes. Git Bash tends to hold files open after you close the window.

  3. Open a Command Window (or Terminal)

    cd to your local repo.

  4. Uninstall lfs

    > git lfs uninstall

    Then it'll say something like:

    Hooks for this repository have been removed.
    Global Git LFS configuration has been removed.
    
  5. Reset

    > git reset --hard

    It'll go through a lot of output probably...

  6. Reinstall lfs

    > git lfs install

    This may again say it found files that should have been pointers but weren't. That's OK, keep going!

  7. Pull with lfs

    > git lfs pull

    Hopefully pulling with lfs will overwrite the files that got borked.

    A few of my sources said at this point their repo was working again, but not me personally. You can open SourceTree or whatever to check if you want, but you may have to start from the top if it didn't work.

  8. Migrate

    The core issue here is that lfs, instead of downloading large files like audio, video, images - anything larger than 1Mb - it just points to them on a server. This is useful if you have a bunch of large files, you're not pulling down all that stuff. So your local repo is smaller and nimbler. However, through circumstances I'm not sure about, it seems possible to corrupt the pointers. I'm sure this is an issue that the lfs people are aware of and are working on, but for now we have to work it out ourselves.

    What we've done so far is

    • uninstall lfs
    • delete everything
    • reinstall lfs
    • pull everything

    So now we have all these things in our folder that are either files or pointers to files, and lfs needs to figure out if any files should be pointers and vise versa. And hopefully by performing the steps above we deleted the corrupted pointers. So we're going to perform migrate to kick off the procedure that goes through the files on the repo, and if they're greater than 1Mb, lfs is going to replace them with a pointer.

    > git lfs migrate

  9. More Errors

    Here's a point at which others have stopped and said they were working again, but not me. I got an error:

    Error in git rev-list... exit status 128 fatal: bad revision '...v1.0.0'

    @guneyozsan over at a github help page, posted this final piece to the puzzle, even though it didn't fix his issue.

    > git lfs migrate info --include-ref=v1.0.0

    Notice the version matches the version that errored - v1.0.0. You will need to replace v1.0.0 with whatever version you got in your error.

    I haven't found a source on why this error occurs but my guess is that the lfs version number generated by migrate on your local repo doesn't match the source version. For me, all this started when SourceTree crashed during a push and I forced a machine reboot, and when that happens, lfs doesn't know how to deal with it, so it just gets stuck in this loop where it's trying to update, but it can't read the corrupted data. Hence the lengthy troubleshooting.

  10. Stash and Pull

When you open SourceTree, you'll probably see that it wants to add all your files back. Don't do that. Stash, then pull.

And boom, the horror is hopefully over. If not, this git hub page or this one may help you more, but this is what worked for me.

1
  • git lfs migrate was a life saver. Thanks!
    – kakyo
    Jul 2, 2021 at 5:45
23

Ensure you have git lfs version 2.5 or higher installed (download here).

Check you are using the git lfs version you downloaded (2.7.2 for me):

>git lfs version
git-lfs/2.7.2

Run:

git lfs migrate import --fixup --everything

Pull your branch and fix any merge conflicts.

Found in this github comment.

1
  • 2
    This is the correct answer, everything else failed for me. Mar 12, 2020 at 13:12
19

run

git add --renormalize .

and commit those changes. It is safe to do even when another user is doing the same on another branch, since the LFS pointer is derived from the hash of the file. It might also catch some files with wrong line endings.

3
  • 3
    I don't know why this is downvoted, it's exactly the correct command to use and it worked for me (just instead of . pass the path to files that need fixing to speed up the process).
    – bgr
    Sep 2, 2021 at 8:34
  • Should be chosen answer. Much less destructive than other approaches.
    – pauljohn32
    May 9, 2022 at 17:27
  • 1
    This approach worked for me to fix files that were both LFS tracked and checked in as the original files. After doing this simple command for each file mentioned by git lfs fsck --pointers and then doing git commit then problem was fixed.
    – Eric
    May 21, 2022 at 4:54
7

One possible cause for this error is due to git LFS-related changes to .gitattributes that impacts already added files in a repo.

(I'm not sure of the exact steps to reproduce, but the issue seemed to occur when I touched a file that was newly affected by the .gitattributes that was previously committed as a non-LFS file that should now be an LFS file. The issue seemed aggravated by switching branches, or at least made it impossible to switch branches until the issue was resolved.)

In this case, I used the steps below to prevent this error from occurring repeatedly.


  1. Fix the issue for the branch you are on by following one of the other answers here (e.g. to clear the cache and reset. I found BheeMa's answer to be effective.)
  2. Go to your main branch, and ensure there is nothing to commit with git status
  3. Force git to recheck and "reapply" git attributes changes

From Ratata Tata's answer in How to make change to .gitattributes take effect)

 git rm --cached -r .
 git add -A

Warning: make sure in step 2 that there was nothing to commit, as the steps above will add any files that were not previously versioned

  1. Verify the results with git status (it should only have modified relevant files to become LFS pointers, i.e. files that can potentially cause the "encountered files that should have been pointers" error) and commit changes
  2. (Optionally, merge/rebase this fix to all other branches if possible. Otherwise, this error could pop up again when switching to those branches. Note that it may be necessary to repeat the initial fix for each branch as per step 1 to be safe, though it could be ok just to commit the affected files.)
5

This just shows one more time what a pile of dog**** GIT-LFS is.

You can get into this situation if:

  1. The file is contained in common-base-branch and not in LFS
  2. In a branch lfs-branch based on common-base-branch, the file was moved to LFS
  3. In another branch non-lfs-branch also based on common-base-branch, the file was modified.

Or alternatively:

  1. The file is NOT contained in common-base-branch
  2. In a branch lfs-branch based on common-base-branch, the file was added to LFS
  3. In another branch non-lfs-branch also based on common-base-branch, the file was added (but not to LFS.

In both cases, when you try to merge non-lfs-branch into lfs-branch, you get this kind of error.

You may ask why this should even happen in the first place, but the answer is that a lot of software is developed by more than once person (which is why you have version control systems like GIT in the first place), and people don't always talk to each other, or LFS is introduced later in the history of a project in a feature branch, while "normal" development still goes on in other branches.

This is a legitimate merge conflict situation, not a bug or corrupted working directory or anything (as some of the other answers suggest). GIT-LFS just handles it poorly.

What you want to do now is to make sure the right version of the conflicted files goes into GIT-LFS, so you may want to choose an answer to this question which does just that... (TODO: Insert link to at least one answer that works)

4

The accepted answer worked for me but it would only work when I manually typed the commands, I put sleeps between each command and now it works as a bash script:

git rm --cached -r .
sleep 1
git reset --hard
sleep 1
git rm .gitattributes
sleep 1
git reset .
sleep 1
git checkout .
3

If you just want to get away from that bad commit, you can go back to master by

git reset --soft origin/master
git reset --hard

Then you are free from the nasty 7 non-LFS files:-)

3

Same as @John Kugelman above, but I put it in a alias because I had to do it a bunch of times.

    
git rm --cached -r . > /dev/null && git reset --hard > /dev/null && git rm .gitattributes > /dev/null && git reset . && git checkout . > /dev/null

1
  • This man is my hero of the day. 🦸‍♂️ Oct 8, 2022 at 3:12
2

The following process will add a commit which replaces all binary files that should be lfs pointers with lfs pointers.

  1. Clean working copy completely. Together with the force add below this prevents any files getting added or removed due to .gitignore patterns.

    git clean -dfx
    git reset --hard
    git checkout -- .
    
  2. Add remove for everything to staging area. Working copy will not be touched.

    git rm --cached -r .
    
  3. Readd all files from working copy again. This will basically undo the previous command but will reevaluate lfs filters. Use -f to ignore .gitignore. All files present were previously checked in and should get added again.

    git add -f .
    
  4. You staging area now should only contain the binary files that previously raised the 'should have been pointers' error.

    git commit -m "moved files to lfs"
    
1
  • I managed to make this work, but for me, I had to make a git commit between the git rm and the git add, otherwise gitk would indicate that I had just re-added the files as binaries and not LFS pointers. Maybe gitk was confused, but to avoid any risks, the two-commit solution seemed more reliable.
    – anol
    Nov 25, 2022 at 8:48
2

Here's the problem I ran into:

Say you created a branch, and you somehow committed files as non-LFS. So then you tried to correct it by later committing the LFS version of the files on the same branch. However, now you can't rebase or squash because you'll keep running into this "files that should have been pointers but weren't" error in the middle of the rebase.

Solve using git reset --soft: https://stackoverflow.com/a/5201642/2516916

2

In my case it was one file under lfs rules (I'm assuming it was checked in without lfs installed or something).

So I found it's extension in the .gitattributes file and commented this line out like

#*.7z filter=lfs diff=lfs merge=lfs -text

saved this .gitattributes then git status showed no problems.

After that I've uncommented this line (removed #), saved .gitattributes and git status still shows no problems.

2

What helped me, without touching the whole repo, was the git restore command which was added in git 2.23

git restore --source=HEAD --staged --worktree -- affected_files

Execute this command a few times until all of the warnings are gone.

1

none of the above commands worked for me, I found an answer here

git status -s | cut -c 4- | xargs git update-index --assume-unchanged
rm .git/index && git reset
3
  • Thank you! None of the other answers worked for me either except this. Mar 2, 2021 at 5:09
  • Why this may or may not work, you may want to explain what this does (the link goes to a website in Japanese). Whether or not this is the right solution may depend on the cause of the problem. Mar 31, 2021 at 11:14
  • 1
    @FlorianWinter the cause of the problem is described in the question above, another reason that this error happens is some people in the team have not enabled git-lfs,so this command have just updated the index , and when you have many files to fix, we used this command: git status -s | cut -c 4- | to collect all the files, I just wrote the answer that worked for me in purpose to help the people who faced the same troubles as me
    – ladhari
    Apr 1, 2021 at 1:15
1

Analysis

That's because the files are not tracked by LFS, but they matches some .gitattributes files' description.

For example,

server/.gitattributes:

conf/** filter=lfs diff=lfs merge=lfs -text
  • server/conf/client.conf is too large and is tracked by LFS
  • server/conf/client.gflags is tracked in git instead of LFS

However, client.gflags matches the server/.gitattributes description, and git will pull it from LFS, but it doesn't have the LFS info, and the error will be thrown up.

Solution

Find the .gitattributes file whose description hit the Encoutered file, delete the wrong description or optimize some wildcard character match.

Optimize the example above, server/.gitattributes:

conf/client.conf filter=lfs diff=lfs merge=lfs -text
1

In addition to the karyon's answer, after performing migrate to fix the local state:

git lfs migrate import --no-rewrite "broken file.jpg" "another broken file.png" ...

You have and extra commit which you probably want to get rid of.

Considering .gitattributes is already fixed on you main / master branch, one can just rebase the local branch onto the one containing the fix, usually:

git rebase upstream/master

And this extra commit will be dropped automatically, since it is meaningless at this point.

0

When it is obviously an error that shows up out of nowhere, this is what we do in our team:

  1. Disable lfs for that specific type file (modifying .gitattributes or via SourceTree menus)

  2. The change will dissapear and you will see a change on .gitattributes instead

  3. Remove the problem:

    3.1 One solution is to execute git reset --hard. Another way, discard changes. Sometimes the file will not come up again.

    3.2.1 If the previous solution doesn't work, repeat 1 and 2. Then make sure that this branch you are in (A) has already commited and pushed everything except those annoying files. Then commit your change, but not push.

    3.2.2: Go to another branch (B)

    3.2.3: Remove that local branch (A) where you performed the commit to .gitattributes, forcing the removal even when it says there's a commit that hasn't been pushed. It will forget that commit (it can afterwards be removed via GC or whatever but it's not a big deal if the file that has the error is not huge)

    3.2.4: Checkout the branch A again. It will download the previous status of the repository without the annoying files and LFS settings set up the correct way.

That always works!

0

Loads of answer on this with multiple steps to fix.

If you are just in a broken state, like stuck on a branch because you can't reset/discard the changes to the problem file(s): deleting the .gitattributes file maybe enough to allow you to make your next git move. Once you have made your git move you may then have to restoring the .gitattributes file but at least you are unstuck.

I wished I had known this before trying all the above. It is a low risk option to try at least.

0

Here with this embellishment that doesn't require reinstalling the lfs hooks or obscuring any .gitattributes files:

git -c filter.lfs.smudge= -c filter.lfs.clean= reset --hard

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.