2

I did a fix on my .gitignore but it took a few commits - I now need to squash them but rebasing ended up erroneous and the rebase was aborted

I tried rebasing with changing pick to squash, using sourcetree and using -f

`git rebase -i e9147 -f
error: The following untracked working tree files would be overwritten by checkout:
        .vscode/settings.json
        DatingApp.API/.vscode/launch.json
        DatingApp.API/.vscode/tasks.json
Please move or remove them before you switch branches.
Aborting
could not detach HEAD`

How can I rebase this correctly?

1 Answer 1

2

Fundamentally, the problem here is that you have several files that are untracked in your current commit, but won't be untracked if Git lets your interactive-rebase proceed.

This situation—at least, the immediate problem, even before any rebase magic is to be applied—is equivalent to the case where you have untracked files but want to git checkout some other branch in which those same files are tracked. For more about this, see Unable to switch branches in git, shows error: The following untracked working tree files would be overwritten by checkout and git says "The following untracked working tree files would be overwritten by checkout" when switching branches. Many of the solutions to these, though, are to deliberately being tracking those files, and commit them now, so that it becomes safe to remove them. It seems likely that .vscode/settings.json, at least, really should not be tracked (and hence not committed).

This leads to a dilemma outlined in this answer: if you do somehow make the rebase happen, you'll probably have these files tracked, even though you don't want want them tracked. If you git add them and commit now, you'll be able to complete the rebase—perhaps with conflicts on these files that should not have been committed—and you can then remove them, but that just moves the problem to the next time someone needs to rebase or otherwise make use of all the commits that do have the files.

For the rest of this answer, I assume that these files should be untracked. Note that the moment you check out a commit that has the files, the files are tracked! (See below for the precise definition of an untracked file, and how, when, and why this gets complicated.)

There is no perfect solution to this problem. My personal preference, though, is to follow the recommendation that Git itself just printed:

Please move or remove them before you switch branches

Take the three files that are untracked—that exist in your work-tree right now, but do not exist in your index right now—and move them out of your work-tree. Now they simply don't exist, as far as Git is concerned. You can complete the rebase. Then, having completed the rebase, which probably results in those three files existing as tracked files now, you can remove the three files (if they're there) and commit the removal (if you had to remove them), so as to make them untracked. Then you just move the saved files—the ones you saved back at the start by moving them to where Git could not see them—back into place as untracked files, and you're back to the file situation you had when you started the rebase, but the rebase is now done.

Long: what exactly is an untracked file and why does this happen?

The definition of an untracked file is really very simple, but it has Git-jargon words in it. An untracked file is a file that exists in your work-tree, but not in your index. This means you need to know what your index and work-tree are. (If you already know, you can stop reading here.)

Git is mainly about commits. Each commit, identified by its own unique, big, ugly, hash ID, stores a snapshot of all of your files, plus some metadata such as who made the commit, when, and why (the log message). All of this stuff is stored in a compressed, read-only, frozen form: no part of any commit can ever be changed. This is pretty great for archives: not only can't you change your stored-forever files, neither can Git, and Git will automatically detect any errors that occur where some content got changed or modified somehow: disk failure, or rogue program that smashed your data, or whatever else might go wrong.

(If you do ever find a bad repository, Git isn't very good at fixing it, but note that Git encourages you to keep dozens of clones lying around. For instance, if you send your Git repository over to GitHub, GitHub have a copy, and you have a copy. If someone has forked your GitHub repository, they have at least one, and probably two, more copies, and so on. With hundreds of copies, chances are that most of them are fine.)

There are lots of good reasons for Git to keep files in frozen, compressed, read-only, Git-only form—I like to call this freeze-dried—inside all the saved commits. We don't need to go into them here; we just need to know that all of your files, plus the rest of the commit metadata, are frozen forever. They can't be changed, and because only Git understands this freeze-dried form, they're not even useful to any non-Git program either. So Git has to provide a way to extract all the files from a commit.

You're already familiar with this, because you have used git checkout. What git checkout does, among other things, is to take the freeze-dried copies of the files that are in some commit, and rehydrate them: convert them into ordinary files that the rest of your computer programs can use. These files are read/write and are good for everything except Git.

The commit is a snapshot of all of your files—well, all of the ones that are in that commit, of course. Git takes all the files that are in the snapshot and restores them to useful form, in what Git calls your work-tree or working tree, or something along these lines. Here, your files are useful again—well, to you anyway; they don't really do much for Git.

But, to get those files into your work-tree, Git first loads all the freeze-dried copies into a place that sits between the current frozen commit—the HEAD commit—and the work-tree. This place is what Git calls, variously, the index, or the staging area, or (rarely) the cache. These three names all refer to the same thing: a holding area for freeze-dried files.

The index, or staging area, actually has a lot of roles, but its number one role is that it holds your proposed next commit. That is, it has all of your files, in their freeze-dried form. You then edit some work-tree file to change it. That edit won't be in the next commit—not yet!—until you git add the file. What git add does is take the work-tree copy, the one you just edited, and re-compress it down to the freeze-dried format. The new, updated freeze-dried file goes into your index / staging area, overwriting the old one. Now the new and updated file will be in the next commit: you've changed your proposed commit to use the new copy (already freeze-dried) instead of the old one.

(This is one reason git commit is so fast, relative to other older version control systems. All git commit has to do is package up the pre-freeze-dried files. Other VCSes, at commit time, do not have an index with a ready-to-go commit. They must gather up every file from your work-tree, turn them into whatever format is used internally, and commit that; and this process is slow, taking multiple seconds or even minutes. Git doesn't even have to look at your work-tree: the index is always ready to go.)

The upshot of all of this is that, for most files, there are three active copies all at the same time. There is a frozen-for-all-time copy in the current commit, that you checked out to get where you are now. There's a second copy, pre-freeze-dried but not actually read-only, in your index. And, of course, there's a useful copy in your work-tree: that's the only one you can see and use directly.

Note that when you run git status, Git runs two separate comparisons:

  • The first one compares HEAD to the index. Whatever files are different here, Git tells you that those files are staged for commit. The rest of your files are still in both HEAD and the index. Git just doesn't say anything about them, because the fact that they're the same isn't very interesting.

  • The second git status comparison is of the index vs the work-tree. Whatever files are different here, Git tells you that those files are not staged for commit. For the rest of the files—the ones that are the same anyway—Git just doesn't say anything.

When you make a new commit, Git will package up everything in the index—all the freeze-dried files—and add the metadata and you have a new commit that saves these files forever (or as long as the new commit exists anyway). But—here come the untracked files!—what happens if you have a file in your work-tree, such as .vscode/settings.json, and it's not in the index?

Well, if a file isn't in the index, it won't be in the next commit. So if you git commit right now, the new commit won't have .vscode/settings.json in it. The file will continue to be in your work-tree, and not in your index.

That's what an untracked file is. An untracked file is one that is in the work-tree, and is not in the index.

We can put files into the index that were not there before: all we have to do is use git add. If you git add a file, well, now it's in the index. If it was there before, you've replaced it with a new verison. If it wasn't there before, you've put the file into the index. Now the file is tracked.

We can also take files out of the index at any time, using git rm. If you git rm .vscode/settings.json (and the file is in the index), Git will remove the file from both the index and the work-tree. If you git rm --cached .vscode/settings.json, Git will remove the file from the index, but leave the work-tree version alone.

So far so good: git add puts files in, git rm --cached takes them out, and neither one messes with the work-tree copy (as long as you remember the --cached). Here's the problem, though: git checkout also puts files into, and takes files out of, the index ... and the work-tree.

Suppose .vscode/settings.json is untracked right now. That means it's in the work-tree, and not in the index. Suppose commit a123456, on some particular branch, has a freeze-dried copy of .vscode/settings.json. This might have the same contents the one in your work-tree, or it might be a different version, but the point is, it's in that commit. If you git checkout that commit, Git will put that version of .vscode/settings.json into your index and put that version of .vscode/settings.json into your work-tree. And now the file is tracked!

Did you want the file to be untracked? Do you still want it to be untracked? Well, it's tracked now, because you did check out a123456. Your only other option is to not check out a123456 after all.

The only ways to deal with this are either to commit the current version of .vscode/settings.json—now it's frozen for all time and you can get it back as long as your new commit continues to exist—or to move it out of the way, so that when Git extracts commit a123456 and overwrites it, well, it's not in the way and not actually being overwritten.

The only other thing that can be done, which is kind of painful, is to "rewrite history". If file .vscode/settings.json should never have been committed in the first place, you can find every commit in the repository that has the file, extract that commit, remove the file, and make a new-and-improved commit that's like the original, but doesn't have the file. Then, having improved all these commits—which in Git also requires improving all their children commits, so it's a pretty big deal—you have to stop using the old commits in favor of the new and improved one.

This isn't that hard—rewriting a whole repository like this is painful, but only one time—but what about all the other clones of this repository? None of them have had their history rewritten. The new history is not really compatible with the old history. You have to get everyone else who has a copy of the repository, to switch over to the new rewritten history too.

That's why there are no good solutions here. Once a file that should always be untracked, has gotten into some commit(s), you must either:

  • be very careful around those commits, or
  • rewrite history and get rid of those commits.

The first one requires care forever; the second one requires that every clone of the repository be fixed-up.

1
  • OK - Thanks for the info - I wasn't really sure what the message was saying - I think after working on it a little more - it was a .gitignore sequencing that reintroduced it into the index - I had to commit, and commit the .gitignore. And somehow the gitignore was off - I managed to get tasks.json out prior to this post but the other three were mystifying. - But I got it working now - my rebase went smoothly. Thanks for the help Aug 7, 2019 at 3:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.