7

Any git clone of a repo above a certain size fails halfway through, I can use ssh method however for this case http is required as it is a secondary action as part of brew install.

$ brew cask install homebrew/cask-versions/adoptopenjdk8

==> Tapping homebrew/cask
Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask'...
remote: Enumerating objects: 422119, done.
error: RPC failed; curl 18 transfer closed with outstanding read data remaining
fatal: the remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
Error: Failure while executing; `git clone https://github.com/Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask` exited with 128.
Follow the instructions here:
  https://github.com/Homebrew/homebrew-cask#reporting-bugs
/usr/local/Homebrew/Library/Homebrew/utils.rb:266:in `safe_system'
/usr/local/Homebrew/Library/Homebrew/tap.rb:273:in `install'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:157:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:123:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:9:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:103:in `<main>'

This is on a brand new machine, with a very fast internet connection (verified the git clone works on other machines). Additionally I have increased buffer size

git config --global http.postBuffer 524288000

I have no problem cloning small repos in both http and ssh method. I have reviewed similar stack questions such as the one below but they don't cover this scenario.

error: RPC failed; curl transfer closed with outstanding read data remaining

3
  • Do you have any proxies, firewalls, antivirus software, SSL/TLS MITM devices, or other network interceptors in place? Also, http.postBuffer has no effect on clones and increasing it will only serve to waste lots of memory when you push. – bk2204 Mar 27 '20 at 1:57
  • No proxies or antivirus, its a new mac out the box. No devices either, but thanks for the heads up on the http.postbuffer ! – Murchie85 Mar 27 '20 at 2:00
  • @Murchie85 Which of the two options I proposed below worked in your case? – VonC Mar 27 '20 at 11:13
8

If the issue persists, meaning if you see the same error message when trying yourself:

git clone https://github.com/Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask

Then try:

  • either an SSH clone:

    git clone git@github.com:Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask
    
  • or a shallow clone as here

    git clone --depth 1 https://github.com/Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask
    cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask
    git fetch --unshallow
    
6
  • 1
    Marked as the right answer, although in my case it did indeed turn out to be anti-virus software that was installed on my ISP end without my knowledge. I realised when switching to ssh and noticed the port was blocked. The methods you have listed work well when there is a poor connection speed. – Murchie85 Mar 27 '20 at 11:14
  • I'm getting the following when trying to unshallow - sudo git fetch --unshallow remote: Enumerating objects: 482427, done. error: RPC failed; curl 18 transfer closed with outstanding read data remaining fatal: the remote end hung up unexpectedly fatal: protocol error: bad pack header – Toby Beresford Oct 1 '20 at 8:33
  • 1
    @TobyBeresford sudo? Nobody makes Git commands as root, usually. What OS and OS version are you using, what Git version are you using? – VonC Oct 1 '20 at 8:36
  • Mac OS X 10.15.6 Catalina - Git 2.23.0. I was trying to overwrite existing partial clones hence the sudo, agree not great practice. :) – Toby Beresford Oct 1 '20 at 8:50
  • @TobyBeresford OK, if the issue persists after upgrading Git to 2.28, you might want to consider posting a separate question, in order to make this more visible than buried in comments. – VonC Oct 1 '20 at 8:54
14

It turns out in my case it was indeed an anti-virus that I wasn't aware of, Virgin Broadband ISP use a built in Virus Safe that I did not know was activated without my consent. For those with this isp, it can be deactivated via https://my.virginmedia.com/my-apps/onlinesecurity/websafe/settings

2
  • 1
    WOW I would never have worked this out -- thank you! – Neil Vass Dec 11 '20 at 22:30
  • I had to turn off all the Home Safe settings from my TalkTalk account – s_diaconu Jan 9 at 17:04
7

&tldr; brew update-reset

I just had the same issue. First I tried @VonC 's answer but it didn't help. I got message

➜  ~ git clone git@github.com:Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask
fatal: destination path '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask' already exists and is not an empty directory.

Googling next for that message gave me this link with solution (in case of mine) of running command

brew update-reset

After that problem was solved

➜  ~ brew update-reset
==> Fetching /usr/local/Homebrew...
remote: Enumerating objects: 61, done.
remote: Counting objects: 100% (34/34), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 7 (delta 4), reused 3 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), 3.55 KiB | 454.00 KiB/s, done.
From https://github.com/Homebrew/brew
 * [new branch]          imgbot     -> origin/imgbot

==> Resetting /usr/local/Homebrew...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'

==> Fetching /usr/local/Homebrew/Library/Taps/dart-lang/homebrew-dart...

==> Resetting /usr/local/Homebrew/Library/Taps/dart-lang/homebrew-dart...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/elastic/homebrew-tap...

==> Resetting /usr/local/Homebrew/Library/Taps/elastic/homebrew-tap...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/heroku/homebrew-brew...

==> Resetting /usr/local/Homebrew/Library/Taps/heroku/homebrew-brew...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask-versions...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask-versions...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
fatal: unable to access 'https://github.com/Homebrew/homebrew-core/': transfer closed with outstanding read data remaining

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-services...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-services...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/sass/homebrew-sass...

==> Resetting /usr/local/Homebrew/Library/Taps/sass/homebrew-sass...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'

Although I see there were one issue with fetching homebrew-core all operation finished successfully and I was able to update packages

➜  ~ brew update && brew upgrade && brew cleanup
Updated 1 tap (homebrew/core).
==> New Formulae
cloudformation-guard
==> Updated Formulae
aws-okta              bitrise               cbmc                  crowdin               frpc                  monero                openrct2              ssh-audit             traefik
==> Upgrading 1 outdated package:
heroku/brew/heroku 7.43.2 -> 7.44.0
==> Upgrading heroku/brew/heroku 7.43.2 -> 7.44.0
==> Downloading https://cli-assets.heroku.com/heroku-v7.44.0/heroku-v7.44.0.tar.xz
######################################################################## 100.0%
==> Caveats
To use the Heroku CLI's autocomplete --
  Via homebrew's shell completion:
    1) Follow homebrew's install instructions https://docs.brew.sh/Shell-Completion
        NOTE: For zsh, as the instructions mention, be sure compinit is autoloaded
          and called, either explicitly or via a framework like oh-my-zsh.
    2) Then run
      $ heroku autocomplete --refresh-cache

  OR

  Use our standalone setup:
    1) Run and follow the install steps:
      $ heroku autocomplete

Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Summary
🍺  /usr/local/Cellar/heroku/7.44.0: 27,477 files, 59.4MB, built in 40 seconds
Removing: /usr/local/Cellar/heroku/7.43.2... (27,475 files, 59.4MB)
Removing: /Users/tpojka/Library/Caches/Homebrew/heroku--7.43.2.tar.xz... (6.9MB)
Removing: /Users/tpojka/Library/Caches/Homebrew/libyaml--0.2.5.catalina.bottle.tar.gz... (107.4KB)
Removing: /Users/tpojka/Library/Caches/Homebrew/nghttp2--1.41.0.catalina.bottle.tar.gz... (968.6KB)
Removing: /Users/tpojka/Library/Logs/Homebrew/htop... (64B)
Removing: /Users/tpojka/Library/Logs/Homebrew/guile... (64B)
Removing: /Users/tpojka/Library/Logs/Homebrew/gnupg... (64B) 

I went one more time with brew update-reset command because of error previously seen and then everything went good with no errors:

➜  ~
➜  ~ brew update-reset
==> Fetching /usr/local/Homebrew...

==> Resetting /usr/local/Homebrew...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/dart-lang/homebrew-dart...

==> Resetting /usr/local/Homebrew/Library/Taps/dart-lang/homebrew-dart...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/elastic/homebrew-tap...

==> Resetting /usr/local/Homebrew/Library/Taps/elastic/homebrew-tap...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/heroku/homebrew-brew...

==> Resetting /usr/local/Homebrew/Library/Taps/heroku/homebrew-brew...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask-versions...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask-versions...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-services...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-services...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/sass/homebrew-sass...

==> Resetting /usr/local/Homebrew/Library/Taps/sass/homebrew-sass...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

➜  ~

I am not quite sure why was this but maybe has something with OS update (happened around that point here). I had notification to upgrade OS to 10.15.7 for a day maybe and initial brew error stood even after updating OS.

3
  • While the update-reset worked, I am still having the RPC failed; curl 18 transfer closed with outstanding read data remaining error when trying to run brew install wget (as an example). :( – Toby Beresford Oct 1 '20 at 8:28
  • 1
    @TobyBeresford Can you tell was it failing on some specific line described in log described in my answer? Do you see any similarities comparing your log with mine? – Tpojka Oct 1 '20 at 8:36
  • 1
    Thanks for asking but I've now found the issue - it was the ISP stopping large downloads from github - adding github.com to my Allowed websites list solved it. :) – Toby Beresford Oct 1 '20 at 8:46
4

It was the Virgin ISP settings. I added https://github.com as an Allowed website: https://my.virginmedia.com/my-apps/onlinesecurity/websafe/settings

Virgin ISP Web Safe Settings

Note that I removed the /usr/local/HomeBrew directory from previous attempts (sudo rm -rf /usr/local/HomeBrew) and then ran /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" as per the standard installation instructions.

And hooray...!

==> Installation successful!

0
2

I manually download the install.sh file, and modify BREW_REPO as BREW_REPO="git@github.com:Homebrew/brew" , then run sh install.sh

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.