555

Some time ago I added info(files) that must be private. Removing from the project is not problem, but I also need to remove it from git history.

I use Git and Github (private account).

Note: On this thread something similar is shown, but here is an old file that was added to a feature branch, that branch merged to a development branch and finally merged to master, since this, a lot of changes was done. So it's not the same and what is needed is to change the history, and hide that files for privacy.

8
  • 3
    You would have to rewrite history. For example git rebase then git push -f Commented May 3, 2017 at 14:07
  • The filter-branch method described in the suggested duplicate will do what you want.
    – 1615903
    Commented May 4, 2017 at 6:00
  • 1
    Also stackoverflow.com/a/17890278 which points to the BFG which can be faster than using git filter-branch
    – Hasturkun
    Commented May 4, 2017 at 6:05
  • but just go faster and do the same, and need to use java i read @Hasturkun Commented May 5, 2017 at 10:16
  • 3
    NONE of theses solutions worked... GIT should have a command for this... it's so useful...
    – marcolopes
    Commented May 11, 2021 at 22:49

12 Answers 12

558

I have found this answer and it helped:

git filter-branch --index-filter 'git rm -rf --cached --ignore-unmatch path_to_file' HEAD
23
  • 69
    Warning: This creates a ton of commits and causes divergence. You probably have to force push after, but I was too scared.
    – sudo
    Commented Aug 27, 2019 at 5:54
  • 7
    Seconding what @sudo said but this did work for my fresh branch that I accidentally committed .env to. Quick and to the point solution.
    – Joe Scotto
    Commented Apr 10, 2020 at 20:16
  • 14
    You can also specify a range of commits as the last argument. If the commit in question was recent, do <previous_hash>..HEAD and save some time. Commented Nov 24, 2020 at 0:53
  • 11
    after this it works only for me git push --force
    – Sebastian
    Commented Jan 4, 2021 at 16:43
  • 34
    Current versions of Git say this about filter-branch: "WARNING: git-filter-branch has a glut of gotchas generating mangled history rewrites. Hit Ctrl-C before proceeding to abort, then use an alternative filtering tool such as 'git filter-repo' (github.com/newren/git-filter-repo) instead. See the filter-branch manual page for more details; to squelch this warning, set FILTER_BRANCH_SQUELCH_WARNING=1."
    – Ryan Lundy
    Commented Nov 28, 2021 at 6:55
268

git-filter-repo

git recommends using the third-party add-on git-filter-repo (when git filter-branch command is executed). There is a long list of reasons why git-filter-repo is better than any other alternatives, my experience is that it is very simple and very fast.

This command removes the file from all commits in all branches:

git filter-repo --invert-paths --path <path to the file or directory>

Multiple paths can be specified by using multiple --path parameters. You can find detailed documentation here: https://www.mankier.com/1/git-filter-repo

24
  • 7
    i get error: git: 'filter-repo' is not a git command. See 'git --help'.
    – cikatomo
    Commented Mar 19, 2021 at 18:51
  • 23
    @cikatomo It's a third-party tool, you have to install it github.com/newren/git-filter-repo/blob/main/INSTALL.md Commented Mar 22, 2021 at 12:10
  • 4
    This worked but it removed .git so I wonder why not just remove .git manually and re-init?
    – chovy
    Commented May 2, 2022 at 11:22
  • 4
    This should be set as the new best answer as it is more up-to-date.
    – GuyStalks
    Commented May 20, 2022 at 13:44
  • 8
    @cikatomo Another way to install pip install git-filter-repo. Commented Aug 29, 2022 at 17:18
181

If you have recently committed that file, or if that file has changed in one or two commits, then I'd suggest you use rebase and cherrypick to remove that particular commit.

Otherwise, you'd have to rewrite the entire history.

git filter-branch --tree-filter 'rm -f <path_to_file>' HEAD

When you are satisfied with the changes and have duly ensured that everything seems fine, you need to update all remote branches -

git push origin --force --all

Note:- It's a complex operation, and you must be aware of what you are doing. First try doing it on a demo repository to see how it works. You also need to let other developers know about it, such that they don't make any change in the mean time.

16
  • 1
    after rewrite the entire history, for keep the changes to repository (github) what must be done? Commented May 3, 2017 at 14:30
  • 3
    Why does your suggestion use --tree-filter rather than --index-filter like in @PetroFranko's answer?
    – einpoklum
    Commented Jun 24, 2020 at 13:10
  • 5
    holy crap, it worked! I mean it was really really simple. I've done it the hard way before, but this was much easier. Tip: the path needs to be relative.
    – Antebios
    Commented May 8, 2021 at 3:07
  • 1
    thank you for the "if recent, try to rebase" solution - my "problem" file was added on a separate branch in a recent commit, so i could rewrite history using a commit pre-dating the file addition
    – Sandra
    Commented Apr 10 at 10:28
  • 1
    @summerisbetterthanwinter copy it before removing may be?
    – hspandher
    Commented May 21 at 16:13
82

Remove the file and rewrite total history with new commit hashs:

there are two ways:

  1. Using git-filter-branch:
git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch <path to the file or directory>' --prune-empty --tag-name-filter cat -- --all
  1. Using git-filter-repo:
pip3 install git-filter-repo
git filter-repo --path <path to the file or directory> --invert-paths

  • now force push the repo: git push origin --force --all
  • tell your collaborators to rebase git pull --rebase.
11
  • 1
    @alper you need to replace PATH-TO-YOUR-FILE-WITH-SENSITIVE-DATA with the file to remove eg: README.md if you want to remove it.
    – suhailvs
    Commented Sep 4, 2021 at 9:14
  • 10
    For git filter-repo: I am getting following message : Aborting: Refusing to destructively overwrite repo history since this does not look like a fresh clone. (expected freshly packed repo) Please operate on a fresh clone instead. If you want to proceed anyway, use --force.. If I force it I get following: fatal: 'origin' does not appear to be a git repository fatal: Could not read from remote repository.
    – alper
    Commented Sep 6, 2021 at 10:46
  • 4
    git filter-branch approach worked for me on mac, while filter-repo approach was removing remote origin Commented Jan 25, 2022 at 9:28
  • 2
    This worked, but I forgot to back up the file first, and now it's gone. :-(
    – kr37
    Commented Apr 30, 2022 at 15:43
  • 1
    You can specify file path like: "*/a.txt" if you happen to have the same file in different paths or files' path have changed in different commits. Commented Feb 4 at 23:05
55

I read this GitHub article, which led me to the following command (similar to the accepted answer, but a bit more robust):

git filter-branch --force --index-filter "git rm --cached --ignore-unmatch PATH-TO-YOUR-FILE-WITH-SENSITIVE-DATA" --prune-empty --tag-name-filter cat -- --all
5
  • 6
    It works better than accepted answer but It also deletes file in locale. Take copy of it before this if you don't want to waste time for rewrite.
    – Kaepxer
    Commented Jul 19, 2021 at 22:27
  • 1
    this helped me on mac, while filter-repo was removing remote origin Commented Jan 25, 2022 at 9:26
  • my local file didn't get removed maybe because it was entered in .gitignore
    – S.aad
    Commented Feb 8, 2022 at 16:32
  • 2
    don't forget the "git push --force" after all your purging is done.
    – Jim Wilcox
    Commented Aug 4, 2022 at 13:56
  • 1
    Works like a charm, thanks. I also added -r in case of deleting a directory
    – Meir Gabay
    Commented Sep 19, 2022 at 13:55
18
  • First of all, add it to your .gitignore file and don't forget to commit the file :-)

  • You can use this site: http://gitignore.io to generate the .gitignore for you and add the required path to your binary files/folder(s)

  • Once you added the file to .gitignore you can remove the "old" binary file with BFG.


How to remove big files from the repository

You can use git filter-branch or BFG. https://rtyley.github.io/bfg-repo-cleaner/

BFG Repo-Cleaner

an alternative to git-filter-branch.

The BFG is a simpler, faster alternative to git-filter-branch for cleansing bad data out of your Git repository history:

*** Removing Crazy Big Files***

  • Removing Passwords, Credentials & other Private data

Examples (from the official site)

In all these examples bfg is an alias for java -jar bfg.jar.

# Delete all files named 'id_rsa' or 'id_dsa' :
bfg --delete-files id_{dsa,rsa}  my-repo.git

enter image description here

4
  • Is it a third party cleaner?
    – alper
    Commented Sep 3, 2021 at 13:39
  • Is it secure to use?
    – alper
    Commented Sep 4, 2021 at 22:08
  • Indeed, a very "old" tool which is being used by the community for few years. The source is in GitHub so you and the community can browse it.
    – CodeWizard
    Commented Sep 5, 2021 at 13:59
  • I just find out that GitHub does not remove deleted commits in case when users request them to run garbage collector, (stackoverflow.com/questions/34582480/remove-commit-for-good/…). I am just get lost where when we use 3rd party tools like GitHub whatever committed, we will always need to ask them to remove it, which is not cool
    – alper
    Commented Sep 5, 2021 at 16:56
11

Using the bfg repo-cleaner package is another viable alternative to git-filter-branch. Apparently, it is also faster...

3

Remove file(s)

bfg --delete-files YOUR-FILE-WITH-SENSITIVE-DATA

Replace all text listed in passwords.txt wherever it can be found in your repository's history, run:

bfg --replace-text passwords.txt

After that you need topush your changes to GitHub/GitLab/BitBucket

git push --force

More about the BFG tool here

Furthermore, since this technique it will rewrite your repository's history, which changes the SHAs for existing commits, you should alter and any dependent commits. So merge and close all open PRs!

2

The following commands should be applied one by one in each project in order to remove the history for a specific file, but you have to take backup from the project at the beginning, because the file will be deleted

  • git filter-branch --index-filter "git rm --cached --ignore-unmatch ProjectFolderName/src/main/resources/application-prod.properties" HEAD

  • git push origin --force --all

  • git update-ref -d refs/original/refs/heads/master

..........................................................................

  • git filter-branch --index-filter "git rm --cached --ignore-unmatch ProjectFolderName/src/main/resources/application.properties" HEAD

  • git push origin --force --all

  • git update-ref -d refs/original/refs/heads/master

0

I had the same "you must be a on a branch" message. I went to the top of the repo in the git UI, and just below the repo name I found a pull down where I could change the branch from default to main. This fixed the issue for me. HTH. enter image description here

0

This command will delete any zip file in your history.

git filter-branch --force --index-filter "git rm --cached --ignore-unmatch *.zip" --prune-empty --tag-name-filter cat -- --all

Replace zip with your files format.

-2

Add the file pattern to your .gitignore. Example:

*.env

Then remove the cached file information using this command(zsh):

bash -c 'git rm --cached *.env'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.