1069

I accidentally dropped a DVD-rip into a website project, then carelessly git commit -a -m ..., and, zap, the repo was bloated by 2.2 gigs. Next time I made some edits, deleted the video file, and committed everything, but the compressed file is still there in the repository, in history.

I know I can start branches from those commits and rebase one branch onto another. But what should I do to merge the 2 commits so that the big file doesn't show in the history and is cleaned in the garbage collection procedure?

11
  • 4
    Related: Completely remove file from all Git repository commit history.
    – user456814
    Apr 4, 2014 at 0:34
  • 1
    Note that if your large file is in a subdir you'll need to specify the full relative path.
    – Johan
    Jul 23, 2015 at 14:36
  • 1
    Also related help.github.com/en/articles/…
    – frederj
    May 27, 2019 at 19:43
  • 10
    Please have also a look at my answer which uses git filter-repo. You should not longer use git filter-branch as it is very slow and often difficult to use. git filter-repo is around 100 times faster.
    – Donat
    Jun 1, 2020 at 19:50
  • 1
    The answers have lots of good info for complex situations. For the simple case where you added the file then removed it in the very next commit, you could just squash those two commits together.
    – piedar
    Mar 5, 2021 at 17:19

23 Answers 23

828

Use the BFG Repo-Cleaner, a simpler, faster alternative to git-filter-branch specifically designed for removing unwanted files from Git history.

Carefully follow the usage instructions, the core part is just this:

$ java -jar bfg.jar --strip-blobs-bigger-than 100M my-repo.git

Any files over 100MB in size (that aren't in your latest commit) will be removed from your Git repository's history. You can then use git gc to clean away the dead data:

$ git reflog expire --expire=now --all && git gc --prune=now --aggressive

After pruning, we can force push to the remote repo*

$ git push --force

*NOTE: cannot force push a protect branch on GitHub

The BFG is typically at least 10-50x faster than running git-filter-branch, and generally easier to use.

Full disclosure: I'm the author of the BFG Repo-Cleaner.

32
  • 6
    @tony It's worth repeating the entire cloning & clearing procedure to see if the message asking you to pull re-occurs, but it's almost certainly because your remote server is configured to reject non-fast-forward updates (ie, it's configured to stop you from losing history - which is exactly what you want to do). You need to get that setting changed on the remote, or failing that, push the updated repo history to a brand new blank repo. Feb 23, 2014 at 23:09
  • 13
    @RobertoTyley Perfect, you save my time, thanks very much. By the way, maybe should do git push --force after your steps, otherwise the remote repo still not changed.
    – Weiyi
    Jul 22, 2015 at 16:16
  • 4
    +1 to adding git push --force. Also worth noting: force pushes may not be allowed by the remote (gitlab.com doesn't, by default. Had to "unprotect" the branch). Sep 10, 2015 at 15:51
  • 3
    BFG worked an absolute charm for me. Brought a 517mb repo down to 38 Mb in just a few minutes. Nothing else worked for me prior to finding this answer.
    – MitchellK
    Aug 14, 2017 at 13:37
  • 3
    Undocumented issue (mostly) when given a "is repo packed" error. Use git gc on the target repo, then re-execute whatever it was you were doing with BFG. Once that was sorted worked pretty well. Could use more explicit documentation, but then I'm not the quickest learner ;p
    – DaveRGP
    Sep 4, 2017 at 13:40
698

NB: Since this answer was written, git filter-branch has been deprecated and it no longer supported. See the man page for more information.


What you want to do is highly disruptive if you have published history to other developers. See “Recovering From Upstream Rebase” in the git rebase documentation for the necessary steps after repairing your history.

You have at least two options: git filter-branch and an interactive rebase, both explained below.

Using git filter-branch

I had a similar problem with bulky binary test data from a Subversion import and wrote about removing data from a git repository.

Say your git history is:

$ git lola --name-status
* f772d66 (HEAD, master) Login page
| A     login.html
* cb14efd Remove DVD-rip
| D     oops.iso
* ce36c98 Careless
| A     oops.iso
| A     other.html
* 5af4522 Admin page
| A     admin.html
* e738b63 Index
  A     index.html

Note that git lola is a non-standard but highly useful alias. (See the addendum at the end of this answer for details.) The --name-status switch to git log shows tree modifications associated with each commit.

In the “Careless” commit (whose SHA1 object name is ce36c98) the file oops.iso is the DVD-rip added by accident and removed in the next commit, cb14efd. Using the technique described in the aforementioned blog post, the command to execute is:

git filter-branch --prune-empty -d /dev/shm/scratch \
  --index-filter "git rm --cached -f --ignore-unmatch oops.iso" \
  --tag-name-filter cat -- --all

Options:

  • --prune-empty removes commits that become empty (i.e., do not change the tree) as a result of the filter operation. In the typical case, this option produces a cleaner history.
  • -d names a temporary directory that does not yet exist to use for building the filtered history. If you are running on a modern Linux distribution, specifying a tree in /dev/shm will result in faster execution.
  • --index-filter is the main event and runs against the index at each step in the history. You want to remove oops.iso wherever it is found, but it isn’t present in all commits. The command git rm --cached -f --ignore-unmatch oops.iso deletes the DVD-rip when it is present and does not fail otherwise.
  • --tag-name-filter describes how to rewrite tag names. A filter of cat is the identity operation. Your repository, like the sample above, may not have any tags, but I included this option for full generality.
  • -- specifies the end of options to git filter-branch
  • --all following -- is shorthand for all refs. Your repository, like the sample above, may have only one ref (master), but I included this option for full generality.

After some churning, the history is now:

$ git lola --name-status
* 8e0a11c (HEAD, master) Login page
| A     login.html
* e45ac59 Careless
| A     other.html
|
| * f772d66 (refs/original/refs/heads/master) Login page
| | A   login.html
| * cb14efd Remove DVD-rip
| | D   oops.iso
| * ce36c98 Careless
|/  A   oops.iso
|   A   other.html
|
* 5af4522 Admin page
| A     admin.html
* e738b63 Index
  A     index.html

Notice that the new “Careless” commit adds only other.html and that the “Remove DVD-rip” commit is no longer on the master branch. The branch labeled refs/original/refs/heads/master contains your original commits in case you made a mistake. To remove it, follow the steps in “Checklist for Shrinking a Repository.”

$ git update-ref -d refs/original/refs/heads/master
$ git reflog expire --expire=now --all
$ git gc --prune=now

For a simpler alternative, clone the repository to discard the unwanted bits.

$ cd ~/src
$ mv repo repo.old
$ git clone file:///home/user/src/repo.old repo

Using a file:///... clone URL copies objects rather than creating hardlinks only.

Now your history is:

$ git lola --name-status
* 8e0a11c (HEAD, master) Login page
| A     login.html
* e45ac59 Careless
| A     other.html
* 5af4522 Admin page
| A     admin.html
* e738b63 Index
  A     index.html

The SHA1 object names for the first two commits (“Index” and “Admin page”) stayed the same because the filter operation did not modify those commits. “Careless” lost oops.iso and “Login page” got a new parent, so their SHA1s did change.

Interactive rebase

With a history of:

$ git lola --name-status
* f772d66 (HEAD, master) Login page
| A     login.html
* cb14efd Remove DVD-rip
| D     oops.iso
* ce36c98 Careless
| A     oops.iso
| A     other.html
* 5af4522 Admin page
| A     admin.html
* e738b63 Index
  A     index.html

you want to remove oops.iso from “Careless” as though you never added it, and then “Remove DVD-rip” is useless to you. Thus, our plan going into an interactive rebase is to keep “Admin page,” edit “Careless,” and discard “Remove DVD-rip.”

Running $ git rebase -i 5af4522 starts an editor with the following contents.

pick ce36c98 Careless
pick cb14efd Remove DVD-rip
pick f772d66 Login page

# Rebase 5af4522..f772d66 onto 5af4522
#
# Commands:
#  p, pick = use commit
#  r, reword = use commit, but edit the commit message
#  e, edit = use commit, but stop for amending
#  s, squash = use commit, but meld into previous commit
#  f, fixup = like "squash", but discard this commit's log message
#  x, exec = run command (the rest of the line) using shell
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#

Executing our plan, we modify it to

edit ce36c98 Careless
pick f772d66 Login page

# Rebase 5af4522..f772d66 onto 5af4522
# ...

That is, we delete the line with “Remove DVD-rip” and change the operation on “Careless” to be edit rather than pick.

Save-quitting the editor drops us at a command prompt with the following message.

Stopped at ce36c98... Careless
You can amend the commit now, with

        git commit --amend

Once you are satisfied with your changes, run

        git rebase --continue

As the message tells us, we are on the “Careless” commit we want to edit, so we run two commands.

$ git rm --cached oops.iso
$ git commit --amend -C HEAD
$ git rebase --continue

The first removes the offending file from the index. The second modifies or amends “Careless” to be the updated index and -C HEAD instructs git to reuse the old commit message. Finally, git rebase --continue goes ahead with the rest of the rebase operation.

This gives a history of:

$ git lola --name-status
* 93174be (HEAD, master) Login page
| A     login.html
* a570198 Careless
| A     other.html
* 5af4522 Admin page
| A     admin.html
* e738b63 Index
  A     index.html

which is what you want.

Addendum: Enable git lola via ~/.gitconfig

Quoting Conrad Parker:

The best tip I learned at Scott Chacon’s talk at linux.conf.au 2010, Git Wrangling - Advanced Tips and Tricks was this alias:

lol = log --graph --decorate --pretty=oneline --abbrev-commit

This provides a really nice graph of your tree, showing the branch structure of merges etc. Of course there are really nice GUI tools for showing such graphs, but the advantage of git lol is that it works on a console or over ssh, so it is useful for remote development, or native development on an embedded board …

So, just copy the following into ~/.gitconfig for your full color git lola action:

[alias]
        lol = log --graph --decorate --pretty=oneline --abbrev-commit
        lola = log --graph --decorate --pretty=oneline --abbrev-commit --all
[color]
        branch = auto
        diff = auto
        interactive = auto
        status = auto
15
  • 5
    Why i can't push when using git filter-branch, failed to push some refs to 'git@bitbucket.org:product/myproject.git' To prevent you from losing history, non-fast-forward updates were rejected Merge the remote changes before pushing again. Feb 4, 2013 at 10:49
  • 11
    Add the -f (or --force) option to your git push command: “Usually, the command refuses to update a remote ref that is not an ancestor of the local ref used to overwrite it. This flag disables the check. This can cause the remote repository to lose commits; use it with care.”
    – Greg Bacon
    Feb 4, 2013 at 23:47
  • 6
    This is a wonderfully thorough answer explaining the use of git-filter-branch to remove unwanted large files from history, but it's worth noting that since Greg wrote his answer, The BFG Repo-Cleaner has been released, which is often faster and easier to use - see my answer for details. Jan 15, 2014 at 15:09
  • 2
    After I do either of the procedures above, the remote repository (on GitHub) does NOT delete the large file. Only the local does. I force push and nada. What am I missing?
    – 4Z4T4R
    May 13, 2014 at 21:11
  • 1
    this also works on dirs. ... "git rm --cached -rf --ignore-unmatch path/to/dir"...
    – rynop
    Aug 20, 2014 at 16:08
315

NB: Since this answer was written, git filter-branch has been deprecated and it no longer supported. See the man page for more information.


Why not use this simple but powerful command?

git filter-branch --tree-filter 'rm -f DVD-rip' HEAD

The --tree-filter option runs the specified command after each checkout of the project and then recommits the results. In this case, you remove a file called DVD-rip from every snapshot, whether it exists or not.

If you know which commit introduced the huge file (say 35dsa2), you can replace HEAD with 35dsa2..HEAD to avoid rewriting too much history, thus avoiding diverging commits if you haven't pushed yet. This comment courtesy of @alpha_989 seems too important to leave out here.

See this link.

9
  • 7
    Much better than bfg. I was unable to clean file from a git with bfg, but this command helped
    – podarok
    Jul 1, 2016 at 11:56
  • 4
    This is great. Just a note for others that you'll have to do this per branch if the large file is in multiple branches.
    – James
    Aug 19, 2016 at 1:38
  • 1
    This worked for me on a local commit that I couldn't upload to GitHub. And it seemed simpler than the other solutions.
    – Richard G
    Feb 3, 2017 at 16:32
  • 6
    If you know the commit where you put the file in (say 35dsa2), you can replace HEAD with 35dsa2..HEAD. tree-filter is much slower than index-filter that way it wont try to checkout all the commits and rewrite them. if you use HEAD, it will try to do that.
    – alpha_989
    Jan 21, 2018 at 20:10
  • 11
    After running the above command, you then have to run git push --all --force to get remote's history to match the amended version you have now created locally (@stevec)
    – Noel Evans
    Jun 16, 2020 at 19:05
117

100 times faster than git filter-branch and easier to use

There are very good answers in this thread, but meanwhile many of them are outdated. Using git-filter-branch is no longer recommended, because it is difficult to use and awfully slow on big repositories with many commits.

git-filter-repo is much faster and easier to use.

git-filter-repo is a Python script, available at github: https://github.com/newren/git-filter-repo . When installed it looks like a regular git command and can be called by git filter-repo.

You need only one file: the Python3 script git-filter-repo. Copy it to a path that is included in the PATH variable. On Windows you may have to change the first line of the script (refer INSTALL.md). You need Python3 installed installed on your system, but this is not a big deal.

First you can run

git filter-repo --analyze

This helps you to determine what to do next.

You can delete your DVD-rip file everywhere:

git filter-repo --invert-paths --path-match DVD-rip
 

Filter-repo is really fast. A task that took around 9 hours on my computer by filter-branch, was completed in 4 minutes by filter-repo. You can do many more nice things with filter-repo. Refer to the documentation for that.

Warning: Do this on a copy of your repository. Many actions of filter-repo cannot be undone. filter-repo will change the commit hashes of all modified commits (of course) and all their descendants down to the last commits!

6
  • 2
    How do I submit the applied changes (on my local repository) to a remote repository? Or this is not possible, and I should clone the amended repo to a new one?
    – diman82
    Feb 1, 2021 at 15:15
  • 4
    @diman82: Best would be to make a new empty repository, set the remote repository from your cloned repo to that and push. This is common to all these answers here: You will get many new commit hashes. This is unavoidable because the commit hashes guarantee for the content and the history of a repo. The alternative way is dangerous, you could make a force push and then run gc to get rid of the files. But do not do this unless you have tested very well and you are aware of all the consequences !
    – Donat
    Feb 1, 2021 at 19:17
  • 17
    git filter-repo --strip-blobs-bigger-than 10M worked much better on my end
    – Lucas
    Jul 13, 2021 at 6:19
  • 1
    This worked well for me. filter-repo has good documentation for more advanced cases but in mine, I just needed to get rid of big file I accidentally committed. In my case, it worked fine to duplicate the project dir, run the command in the new version, re-add the remote and push (no strictly fresh clone).
    – Alex L
    May 24, 2022 at 22:25
  • 3
    this should be the accepted answer now. Worked amazingly well.
    – james-see
    May 31, 2022 at 20:24
110

After trying virtually every answer in SO, I finally found this gem that quickly removed and deleted the large files in my repository and allowed me to sync again: http://www.zyxware.com/articles/4027/how-to-delete-files-permanently-from-your-local-and-remote-git-repositories

CD to your local working folder and run the following command:

git filter-branch -f --index-filter "git rm -rf --cached --ignore-unmatch FOLDERNAME" -- --all

replace FOLDERNAME with the file or folder you wish to remove from the given git repository.

Once this is done run the following commands to clean up the local repository:

rm -rf .git/refs/original/
git reflog expire --expire=now --all
git gc --prune=now
git gc --aggressive --prune=now

Now push all the changes to the remote repository:

git push --all --force

This will clean up the remote repository.

9
  • 1
    Worked like a charm for me. Apr 16, 2018 at 7:17
  • 4
    This worked for me as well. Gets rid of a specific folder (in my case, one that contained files too large or a Github repo) on the repository, but keeps it on the local file system in case it exists.
    – skizzo
    Jul 8, 2018 at 12:13
  • Worked for me! no history is left which is potentially confusing (if someone where to clone right now), make sure you have a plan to update any broken links, dependencies, etc Jun 19, 2019 at 5:11
  • I tried the filter-branch methods described in the other answers, but they didn't work. After filtering, I still got file size too big error when pushing to GitHub. This solution worked, most likely because it removed the big file from ALL occurrences in ALL branches. Jul 30, 2020 at 22:43
  • 1
    I guess it's only me that didn't realize this command will also nuke the file from the project itself, not just the git repo. Certainly worked though!
    – Karl
    Nov 16, 2021 at 2:38
44

These commands worked in my case:

git filter-branch --force --index-filter 'git rm --cached -r --ignore-unmatch oops.iso' --prune-empty --tag-name-filter cat -- --all
rm -rf .git/refs/original/
git reflog expire --expire=now --all
git gc --prune=now
git gc --aggressive --prune=now

It is little different from the above versions.

For those who need to push this to github/bitbucket (I only tested this with bitbucket):

# WARNING!!!
# this will rewrite completely your bitbucket refs
# will delete all branches that you didn't have in your local

git push --all --prune --force

# Once you pushed, all your teammates need to clone repository again
# git pull will not work
3
  • 4
    How is it different from above, why is it better? Jun 14, 2013 at 9:08
  • 1
    For some reason mkljun version is not reduced git space in my case, I already had removed the files from index by using git rm --cached files. The Greg Bacon's proposition is more complete, and quite the same to this mine, but he missed the --force index for cases when you are using filter-branch for multiple times, and he wrote so much info, that my version is like resume of it.
    – Kostanos
    Jun 14, 2013 at 14:09
  • 1
    This really helped but I needed to use the -f option not just -rf here git rm --cached -rf --ignore-unmatch oops.iso instead of git rm --cached -r --ignore-unmatch oops.iso as per @lfender6445 below
    – drstevok
    Oct 21, 2016 at 6:18
19

According to GitHub Documentation, just follow these steps:

  1. Get rid of the large file

Option 1: You don't want to keep the large file:

rm path/to/your/large/file        # delete the large file

Option 2: You want to keep the large file into an untracked directory

mkdir large_files                       # create directory large_files
touch .gitignore                        # create .gitignore file if needed
'/large_files/' >> .gitignore           # untrack directory large_files
mv path/to/your/large/file large_files/ # move the large file into the untracked directory
  1. Save your changes
git add path/to/your/large/file   # add the deletion to the index
git commit -m 'delete large file' # commit the deletion
  1. Remove the large file from all commits
git filter-branch --force --index-filter \
  "git rm --cached --ignore-unmatch path/to/your/large/file" \
  --prune-empty --tag-name-filter cat -- --all
git push <remote> <branch>
5
  • can you elaborate on how the "remove the large file from all commits" step worked, that was amazing!
    – clayg
    Dec 2, 2020 at 21:51
  • Thanks @clayg. I don't understand deeply the git filter-branch command, as I wrote, I just followed the GitHub documentation. What I know is that this command browses through your .git folder and find all tracks of the given file and removes it from the history.
    – Kevin R.
    Dec 28, 2020 at 10:10
  • @KevinR. you have to force push, isnt it?
    – Exploring
    Apr 29, 2022 at 1:54
  • That is correct @Exploring
    – Kevin R.
    Nov 24, 2022 at 9:10
  • Thank you, damn python programmers and checking in binary files.
    – Owl
    Apr 5 at 13:04
13

I ran into this with a bitbucket account, where I had accidentally stored ginormous *.jpa backups of my site.

git filter-branch --prune-empty --index-filter 'git rm -rf --cached --ignore-unmatch MY-BIG-DIRECTORY-OR-FILE' --tag-name-filter cat -- --all

Relpace MY-BIG-DIRECTORY with the folder in question to completely rewrite your history (including tags).

source: https://web.archive.org/web/20170727144429/http://naleid.com:80/blog/2012/01/17/finding-and-purging-big-files-from-git-history/

2
  • 1
    This response helped me, except the script in the answer has a slight issue and it doesn't search in all branches form me. But the command in the link did it perfectly.
    – Ali B
    Sep 5, 2015 at 20:20
  • Add -f after git filter-branch, if need to overwrite previous backup
    – Sheldon
    Jun 1, 2022 at 9:32
10

Just note that this commands can be very destructive. If more people are working on the repo they'll all have to pull the new tree. The three middle commands are not necessary if your goal is NOT to reduce the size. Because the filter branch creates a backup of the removed file and it can stay there for a long time.

$ git filter-branch --index-filter "git rm -rf --cached --ignore-unmatch YOURFILENAME" HEAD
$ rm -rf .git/refs/original/ 
$ git reflog expire --all 
$ git gc --aggressive --prune
$ git push origin master --force
3
  • 14
    Do NOT run these commands unless you want to create immense pain for yourself. It deleted a lot of my original source code files. I assumed it would purge some large files from my commit history in GIT (as per the original question), however, I think this command is designed to permanently purge files from your original source code tree (big difference!). My system: Windows, VS2012, Git Source Control Provider.
    – Contango
    Oct 22, 2012 at 11:16
  • 2
    I used this command: git filter-branch --force --index-filter 'git rm --cached -r --ignore-unmatch oops.iso' --prune-empty --tag-name-filter cat -- --all instead of first one from your code
    – Kostanos
    Jun 14, 2013 at 2:31
  • 2
    @mkljun, please at least remove "git push origin master --force"! First of all it is not related to the original question - author didn't ask how to edit commits and push changes to some repository. And second - this is dangerous, you really can delete a lot of files and push changes to remote repository without first check what was deleted is not a good idea.
    – Ezh
    Aug 21, 2021 at 10:27
9

git filter-branch --tree-filter 'rm -f path/to/file' HEAD worked pretty well for me, although I ran into the same problem as described here, which I solved by following this suggestion.

The pro-git book has an entire chapter on rewriting history - have a look at the filter-branch/Removing a File from Every Commit section.

8

NEW ANSWER THAT WORKS IN 2022.

DO NOT USE:

git filter-branch

this command might not change the remote repo after pushing. If you clone after using it, you will see that nothing has changed and the repo still has a large size. It seems this command is old now. For example, if you use the steps in https://github.com/18F/C2/issues/439, this won't work.

The Solution

This solution is based on using:

git filter-repo

Steps:

(1) Find the largest files in .git (change 10 to whatever number of files you wanna display):

git rev-list --objects --all | grep -f <(git verify-pack -v  .git/objects/pack/*.idx| sort -k 3 -n | cut -f 1 -d " " | tail -10)

(2) Start filtering these large files by passing the path&name of the file you would like to remove:

 git filter-repo --path-glob '../../src/../..' --invert-paths --force

or use the extension of the file e.g. to filter all zip files:

 git filter-repo --path-glob '*.zip' --invert-paths --force

or e.g. to filter all .a lib files:

 git filter-repo --path-glob '*.a' --invert-paths --force

or whatever you find in step 1.

(3)

 git remote add origin git@github.com:.../...git

(4)

git push --all --force

git push --tags --force

DONE!!!

3
  • What does "Strat" mean in item 2). What are you doing in that step. Please explain what 3 is doing, especially ".../...git". I already have repo with a remote. What is all of the .../ about?
    – pauljohn32
    Nov 30, 2022 at 0:36
  • I like this solution. Poster should've mentioned "filter-repo" isn't a native git command, you have to install a python script: github.com/newren/git-filter-repo
    – inorganik
    Feb 1 at 17:12
  • Is this a message from the future? Please tell me what life is like in 20222. I can't believe you are still using git. May 11 at 0:53
8

If you know your commit was recent instead of going through the entire tree do the following: git filter-branch --tree-filter 'rm LARGE_FILE.zip' HEAD~10..HEAD

5

I basically did what was on this answer: https://stackoverflow.com/a/11032521/1286423

(for history, I'll copy-paste it here)

$ git filter-branch --index-filter "git rm -rf --cached --ignore-unmatch YOURFILENAME" HEAD
$ rm -rf .git/refs/original/ 
$ git reflog expire --all 
$ git gc --aggressive --prune
$ git push origin master --force

It didn't work, because I like to rename and move things a lot. So some big file were in folders that have been renamed, and I think the gc couldn't delete the reference to those files because of reference in tree objects pointing to those file. My ultimate solution to really kill it was to:

# First, apply what's in the answer linked in the front
# and before doing the gc --prune --aggressive, do:

# Go back at the origin of the repository
git checkout -b newinit <sha1 of first commit>
# Create a parallel initial commit
git commit --amend
# go back on the master branch that has big file
# still referenced in history, even though 
# we thought we removed them.
git checkout master
# rebase on the newinit created earlier. By reapply patches,
# it will really forget about the references to hidden big files.
git rebase newinit

# Do the previous part (checkout + rebase) for each branch
# still connected to the original initial commit, 
# so we remove all the references.

# Remove the .git/logs folder, also containing references
# to commits that could make git gc not remove them.
rm -rf .git/logs/

# Then you can do a garbage collection,
# and the hidden files really will get gc'ed
git gc --prune --aggressive

My repo (the .git) changed from 32MB to 388KB, that even filter-branch couldn't clean.

4

Use Git Extensions, it's a UI tool. It has a plugin named "Find large files" which finds lage files in repositories and allow removing them permenently.

Don't use 'git filter-branch' before using this tool, since it won't be able to find files removed by 'filter-branch' (Altough 'filter-branch' does not remove files completely from the repository pack files).

4
  • This method is waaay too slow for large repositories. It took over an hour to list the large files. Then when I go to delete files, after an hour it is only 1/3 of the way through processing the first file I want to delete.
    – kristianp
    Oct 4, 2017 at 4:19
  • Yes, its slow, but does the work... Do you know anything quicker?
    – Nir
    Oct 6, 2017 at 21:03
  • 1
    Haven't used it, but BFG Repo-Cleaner, as per another answer on this page.
    – kristianp
    Oct 9, 2017 at 4:42
  • Git Extension is nice and simple. However it uses git filter-branch internally, so deletion is very slow. Nov 10, 2022 at 19:19
3

git filter-branch is a powerful command which you can use it to delete a huge file from the commits history. The file will stay for a while and Git will remove it in the next garbage collection. Below is the full process from deleteing files from commit history. For safety, below process runs the commands on a new branch first. If the result is what you needed, then reset it back to the branch you actually want to change.

# Do it in a new testing branch
$ git checkout -b test

# Remove file-name from every commit on the new branch
# --index-filter, rewrite index without checking out
# --cached, remove it from index but not include working tree
# --ignore-unmatch, ignore if files to be removed are absent in a commit
# HEAD, execute the specified command for each commit reached from HEAD by parent link
$ git filter-branch --index-filter 'git rm --cached --ignore-unmatch file-name' HEAD

# The output is OK, reset it to the prior branch master
$ git checkout master
$ git reset --soft test

# Remove test branch
$ git branch -d test

# Push it with force
$ git push --force origin master
1

When you run into this problem, git rm will not suffice, as git remembers that the file existed once in our history, and thus will keep a reference to it.

To make things worse, rebasing is not easy either, because any references to the blob will prevent git garbage collector from cleaning up the space. This includes remote references and reflog references.

I put together git forget-blob, a little script that tries removing all these references, and then uses git filter-branch to rewrite every commit in the branch.

Once your blob is completely unreferenced, git gc will get rid of it

The usage is pretty simple git forget-blob file-to-forget. You can get more info here

https://ownyourbits.com/2017/01/18/completely-remove-a-file-from-a-git-repository-with-git-forget-blob/

I put this together thanks to the answers from Stack Overflow and some blog entries. Credits to them!

1
  • you should get this in homebrew
    – Cameron E
    May 14, 2017 at 7:54
1

You can do this using the branch filter command:

git filter-branch --tree-filter 'rm -rf path/to/your/file' HEAD

1

This was such a useful comment by @Lucas that I've decided to post it as an answer so that more people see it.

They said to use git-filter-repo and run the command: git filter-repo --strip-blobs-bigger-than 10M

If you're struggling to install git-filter-repo on Windows (like I was), please see this.

What does this do and how does it work? I don't know. If you do, please leave a comment.

However, afterwards, my commit history remained with all the huge files no longer in the commit history. It worked.

As always, back up your repo before running this.

0

Other than git filter-branch (slow but pure git solution) and BFG (easier and very performant), there is also another tool to filter with good performance:

https://github.com/xoofx/git-rocket-filter

From its description:

The purpose of git-rocket-filter is similar to the command git-filter-branch while providing the following unique features:

  • Fast rewriting of commits and trees (by an order of x10 to x100).
  • Built-in support for both white-listing with --keep (keeps files or directories) and black-listing with --remove options.
  • Use of .gitignore like pattern for tree-filtering
  • Fast and easy C# Scripting for both commit filtering and tree filtering
  • Support for scripting in tree-filtering per file/directory pattern
  • Automatically prune empty/unchanged commit, including merge commits
0

I had the same problem. so with git rebase -i HEAD~15 I turned the commit which had large file to edit mode, then git rm {relative/path/largeFile} removed the large file from the commit and did git rebase --continue.

also I added {relative/path/largeFile} filter=lfs diff=lfs merge=lfs -text to .gitattributes and did a commit.

note the git filter-repo eventhough messaged successful didn't work for me. note I cloned the git clone https://github.com/newren/git-filter-repo.git in another directory. then from that directory ran python git-filter-repo --path "{large\File\Path}" --invert-paths.

-1

Save a backup of your current code in case anything goes wrong during this process.

git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch path/to/large_file' --prune-empty --tag-name-filter cat -- --all

Replace path/to/large_file with the actual path to the large file that you want to remove. This command will rewrite the Git history and remove the large file from all commits.

After running the git filter-branch command, you may see a message that says "Ref 'refs/heads/master' is unchanged" or similar. This indicates that the branch is not updated yet. To update the branch and apply the changes, use:

git push origin --force --all
-4

This works perfectly for me : in git extensions :

right click on the selected commit :

reset current branch to here :

hard reset ;

It's surprising nobody else is able to give this simple answer.

reset current branch to here

hard reset

2
  • 1
    Worked for me but me mindful this deletes everything after that point
    – Jossy
    Jul 19, 2020 at 16:22
  • 2
    No-one gave this answer because it does not answer the question. He wants a specific file removed from the history. Your answer nukes everything in the repo after a certain point. Apr 16, 2021 at 22:51
-6
git reset --soft HEAD~1

It will keep the changes but remove the commit then you can re-commit those changes.

Not the answer you're looking for? Browse other questions tagged or ask your own question.