671

I have some files in my repository that should be ignored, i added them to the .gitignore but, of course, they are not removed from my repository.

So my question is, is there a magic command or script using filter-branch that can rewrite my history and remove all these files easily? Or simply a command that will create a commit that will remove them ?

6

13 Answers 13

788

You can remove them from the repository manually:

git rm --cached file1 file2 dir/file3

Or, if you have a lot of files:

git rm --cached `git ls-files -i -c --exclude-from=.gitignore`

But this doesn't seem to work in Git Bash on Windows. It produces an error message. The following works better:

git ls-files -i -c --exclude-from=.gitignore | xargs git rm --cached  

In PowerShell on Windows this works even better (handles spaces in path and filenames):

git ls-files -i -c --exclude-from=.gitignore | %{git rm --cached $_}

Regarding rewriting the whole history without these files, I highly doubt there's an automatic way to do it.
And we all know that rewriting the history is bad, don't we? :)

18
  • 4
    Unfortunately the Git Bash on Windows command doesn't work with paths that contain spaces
    – Nate Bundy
    Apr 23, 2014 at 18:50
  • 1
    @NateBundy if you're referring to the fact that xargs won't work with spaces, most command line utilities get around that by using special flags so that whitespace won't matter. Off the top of my head I don't remember what the flags are for git ls-files and xargs (I think it might be -0 for xargs), but you can look them up.
    – user456814
    May 25, 2014 at 19:50
  • 1
    "git ls-files -i --exclude-from=.gitignore" is very helpful, it tells me what files are excluded by .ignore.
    – Owen Cao
    Jan 25, 2019 at 0:22
  • 1
    I was happy to find these commands, BUT it does not really remove the files from repository. When i removed 2300 kB of images, repository size dropped only 10 kB. So it cannot be used, for example, to make repository smaller and faster to transfer. Feb 22, 2019 at 7:41
  • 1
    Note that regardless of your current directory, --exclude-from= is relative to the git root. So if you want to do this for a .gitignore in a subdirectory, use --exclude-from=[subdirectory]/.gitignore. Feb 2, 2021 at 8:40
670

An easier way that works regardless of the OS is to do

git rm -r --cached .
git add .
git commit -m "Drop files from .gitignore"

You basically remove and re-add all files, but git add will ignore the ones in .gitignore.

Using the --cached option will keep files in your filesystem, so you won't be removing files from your disk.

Be sure to do this from a clean working directory, i.e. commit your other diffs before doing this.

Note: Some pointed out in the comments that you will lose the history of all your files. I tested this with git 2.27.0 on MacOS and it is not the case. If you want to check what is happening, check your git diff HEAD~1 before you push your commit.

This won't remove the .gitignor'ed files from the git history of your repo. If you want to do that, check Ignore files committed to git and also remove them from history.

17
  • 4
    what do you mean by fake commit message? It's a real commit message :P You can change the message of course, depending on your needs...
    – gtatr
    Aug 1, 2019 at 21:15
  • 27
    Don't do this, it will remove history on all the files
    – agrath
    Mar 18, 2020 at 2:50
  • 6
    @agrath where did you test this? Which version of git? Which OS? It is not the case for me on MacOS and git 2.27.0
    – gtatr
    Dec 4, 2020 at 9:02
  • 3
    Just to clarify a bit because the git status after doing git rm -r --cached . git add . can look a bit scary: These three commands that @gtatr provides essentially delete files from git that were previously tracked, but have since been added to your .gitignore file. When I first ran this, I saw a bunch of files and freaked out for a second but upon further inspection I noticed they were all files that were listed in my .gitignore file. Feb 11, 2021 at 19:54
  • 7
    This should absolutely be the accepted answer, and you don't remote the history of the files (anyway you can check always before committing). Just did against my repo on GitHub, perfectly removed all unwanted files and history is still there! Thanks!
    – bastio84
    Jul 25, 2022 at 11:06
218

As the files in .gitignore are not being tracked, you can use the git clean command to recursively remove files that are not under version control.

Use git clean -xdn to perform a dry run and see what will be removed.
Then use git clean -xdf to execute it.

Basically, git clean -h or man git-clean(in unix) will give you help.

Be aware that this command will also remove new files that are not in the staging area.

5
  • 45
    This answer isn't applicable - the OP says the files in .gitignore are being tracked. Dec 15, 2016 at 23:15
  • 48
    BEWARE! This permanently deletes all untracked files, rather than just removing from the branch
    – Emmanuel
    Mar 6, 2018 at 6:38
  • 13
    the git clean -xdn is a dry run which won't delete. the next one will.
    – JohnZaj
    Dec 18, 2018 at 22:30
  • 49
    -1: This is a highly misleading answer - the original poster wanted to remove from the repository, not remove the files completely. I was this close to deleting a load of dynamic files required by my IDE but not required to be in the repo.
    – Auspice
    Dec 30, 2018 at 14:11
  • It was actually helpful for me but missleading. Probably better as a comment to the accepted answer. Jul 19, 2023 at 11:49
9

I did a very straightforward solution by manipulating the output of the .gitignore statement with sed:

cat .gitignore | sed '/^#.*/ d' | sed '/^\s*$/ d' | sed 's/^/git rm -r /' | bash

Explanation:

  1. print the .gitignore file
  2. remove all comments from the print
  3. delete all empty lines
  4. add 'git rm -r ' to the start of the line
  5. execute every line.
4
  • 28
    sed is straightforward? Sep 13, 2018 at 10:52
  • Siilar example with another problem: stackoverflow.com/a/63534476/7127519 Aug 22, 2020 at 9:08
  • +1 for scripting and explaining it. Explaining the danger of running it right into bash without review may have been an oversight.
    – Rawheiser
    Dec 30, 2020 at 19:41
  • This is not robust. Some gitignore files use a whitelist mode where they ignore everything but exclude what needs to be kept using "!" Jun 9, 2023 at 9:24
6

git rm --cached -r . to remove all cached recursively

git add . to add all files not included in .gitignore

you will have to commit some deleted files that are not really deleted on File System

using a single command it will be git rm --cached -r . && git add .

4

"git clean"(man) and git ls-files -i(man) had confusion around working on or showing ignored paths inside an ignored directory, which has been corrected with Git 2.32 (Q2 2021).

That means the 2021 version of the accepted answer would be:

git ls-files -i -c --exclude-from=.gitignore | xargs git rm --cached  
                ^^

See commit b548f0f, commit dd55fc0, commit aa6e1b2, commit a97c7a8, commit 2e4e43a, commit b338e9f, commit 7fe1ffd, commit 7f9dd87 (12 May 2021) by Elijah Newren (newren).
See commit 4e689d8 (12 May 2021) by Derrick Stolee (derrickstolee).
(Merged by Junio C Hamano -- gitster -- in commit 33be431, 20 May 2021)

ls-files: error out on -i unless -o or -c are specified

Signed-off-by: Elijah Newren

ls-files --ignored(man) can be used together with either --others or --cached.

After being perplexed for a bit and digging in to the code, I assumed that ls-files -i was just broken and not printing anything and I had a nice patch ready to submit when I finally realized that -i can be used with --cached to find tracked ignores.

While that was a mistake on my part, and a careful reading of the documentation could have made this more clear, I suspect this is an error others are likely to make as well.
In fact, of two uses in our testsuite, I believe one of the two did make this error.
In t1306.13, there are NO tracked files, and all the excludes built up and used in that test and in previous tests thus have to be about untracked files.
However, since they were looking for an empty result, the mistake went unnoticed as their erroneous command also just happened to give an empty answer.

-i will most the time be used with -o, which would suggest we could just make -i imply -o in the absence of either a -o or -c, but that would be a backward incompatible break.
Instead, let's just flag -i without either a -o or -c as an error, and update the two relevant testcases to specify their intent.

That means without -c, you would get (starting with Git 2.32, Q2 2021):

fatal: ls-files -i must be used with either -o or -c

Note: this is still a work in progress, since it was reverted in Git 2.32-rc2 but fixed with commit 2c9f1bf, commit 1df046b (27 May 2021) by Junio C Hamano (gitster).
See commit 906fc55 (27 May 2021) by Elijah Newren (newren).
See commit eef8148 (27 May 2021) by Derrick Stolee (derrickstolee).
(Merged by Junio C Hamano -- gitster -- in commit 329d63e, 28 May 2021)

dir: introduce readdir_skip_dot_and_dotdot() helper

Signed-off-by: Elijah Newren

2

This solution adds carriage returns (I'm a WSL user, so this is important), and parenthesis escaping (which is important to LaTeX users sometimes, e.g. *.synctex(busy)).


Inspired by Scott's solution:

cat .gitignore | sed "s/\r//" | sed -r "/^(#.*|\s*)$/d" | sed -r "s/([()])/\\\\\1/g" | sed "s/^/git rm -r /" | bash
  1. Remove: carriage returns (s/\r//).
  2. Remove lines containing: comments (/^#.*$/), empty line groups (/^\s*$/, matches whitespace or empty line). Notice the pipe | character, this is standard regex, and requires -r (although I believe -E also works).
  3. Replace: parenthesis /([()])/ with its escaped version \\\1, \1 matches the group, in this case it means [()], or ( or ), whatever was matched. Notice the g flag, this is to match (and replace) all parenthesis. Could be rewritten as "s/(\(|\))/\\\\\1/g" if you're into that.
  4. Prepend git rm -r

Replacement looks like s/$old/$new/$flags. Removal looks like /$old/d. Prepending is replacing /^/. And you could do appending by replacing /$/. And of course, some characters are escaped, since you can't make raw strings in bash as far as I know. Finally, this line can be condensed, but I chose to leave it expanded for the sake of readability.


I saw someone questioning (in Scott's solution) that sed is straight forward. I like to think of this method as the most basic and most monkey-way to do it, this is good, because if you ever need a variation of this, you can make it on the spot. And if anything, it's a good excuse to practice regular expressions.

1

If you really want to prune your history of .gitignored files, first save .gitignore outside the repo, e.g. as /tmp/.gitignore, then run

git filter-branch --force --index-filter \
    "git ls-files -i -X /tmp/.gitignore -c | xargs -r git rm --cached --ignore-unmatch -rf" \
    --prune-empty --tag-name-filter cat -- --all

Notes:

  • https://git-scm.com/docs/git-filter-branch#_warning is not just there for fun
  • git filter-branch --index-filter runs in the .git directory I think, i.e. if you want to use a relative path you have to prepend one more ../ first. And apparently you cannot use ../.gitignore, the actual .gitignore file, that yields a "fatal: cannot use ../.gitignore as an exclude file" for some reason (maybe during a git filter-branch --index-filter the working directory is (considered) empty?)
  • I was hoping to use something like git ls-files -iX <(git show $(git hash-object -w .gitignore)) instead to avoid copying .gitignore somewhere else, but that alone already returns an empty string (whereas cat <(git show $(git hash-object -w .gitignore)) indeed prints .gitignore's contents as expected), so I cannot use <(git show $GITIGNORE_HASH) in git filter-branch...
  • If you actually only want to .gitignore-clean a specific branch, replace --all in the last line with its name. The --tag-name-filter cat might not work properly then, i.e. you'll probably not be able to directly transfer a single branch's tags properly
1
-1

In linux you can use this command:

For example I want to delete *.py~ so my command should be ==>

find . -name "*.py~" -exec rm -f {} \;

-1

For GitHub, the easiest way is to open the online VS Code editor by pressing . when viewing the repo. Then you can just right-click and delete the files/folders in the left hand panel and commit the change.

VS Code Source Control

-1

A simple solution to remove files already committed but now in .gitignore, execute these commands:

git rm --cached `git ls-files -i --exclude-from=.gitignore`
git commit -m "Remove ignored files"

This will remove the files from the repository (keeping them in your working directory) and create a commit for the changes.

-2

Simply go to gitignore file and remove whatever added newly and then commit the changes, it remove contents from there

1
  • 1
    As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Aug 14, 2023 at 18:09
-5

The git will ignore the files matched .gitignore pattern after you add it to .gitignore.

But the files already existed in repository will be still in.

use git rm files_ignored; git commit -m 'rm no use files' to delete ignored files.

1
  • 4
    There are a lot of them, is there a way to delete them without having to specifying their names?
    – Intrepidd
    Nov 24, 2012 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.