392

My git client repeatedly fails with the following error after trying to clone the repository for some time.

What could be the issue here?

Note: I have registered my SSH key with the GIT hosting provider

Receiving objects:  13% (1309/10065), 796.00 KiB | 6 KiB/s
fatal: The remote end hung up unexpectedly
12
  • Can you check if your git hosting provider is online?
    – Caps
    Jul 27, 2011 at 10:16
  • @Caps it is online and the network is fine also. It seems to happen consistently after some time.
    – Joe
    Jul 27, 2011 at 10:28
  • 8
    Can you check if a git config --global http.postBuffer 524288000 has any effect on your clone? It there any additional error message like a 'error: RPC failed; result=56, HTTP code = 0'
    – VonC
    Jul 27, 2011 at 10:41
  • @VonC - The above command executed just fine, didn't see any output on the console.
    – Joe
    Jul 27, 2011 at 10:50
  • 3
    @Joe are you able to clone after the git config --global http.postBuffer 524288000?
    – VonC
    Jul 27, 2011 at 10:54

43 Answers 43

639

Quick solution:

With this kind of error, I usually start by raising the postBuffer size by:

git config --global http.postBuffer 524288000

(some comments below report having to double the value):

git config --global http.postBuffer 1048576000

(For npm publish, Martin Braun reports in the comments setting it to no more than 50 000 000 instead of the default 1 000 000)

More information:

From the git config man page, http.postBuffer is about:

Maximum size in bytes of the buffer used by smart HTTP transports when POSTing data to the remote system.
For requests larger than this buffer size, HTTP/1.1 and Transfer-Encoding: chunked is used to avoid creating a massive pack file locally. Default is 1 MiB, which is sufficient for most requests.

Even for the clone, that can have an effect, and in this instance, the OP Joe reports:

[clone] works fine now


Note: if something went wrong on the server side, and if the server uses Git 2.5+ (Q2 2015), the error message might be more explicit.
See "Git cloning: remote end hung up unexpectedly, tried changing postBuffer but still failing".


Kulai (in the comments) points out to this Atlassian Troubleshooting Git page, which adds:

Error code 56 indicates a curl receive the error of CURLE_RECV_ERROR which means there was some issue that prevented the data from being received during the cloning process.
Typically this is caused by a network setting, firewall, VPN client, or anti-virus that is terminating the connection before all data has been transferred.

It also mentions the following environment variable, order to help with the debugging process.

# Linux
export GIT_TRACE_PACKET=1
export GIT_TRACE=1
export GIT_CURL_VERBOSE=1

#Windows
set GIT_TRACE_PACKET=1
set GIT_TRACE=1
set GIT_CURL_VERBOSE=1

With Git 2.25.1 (Feb. 2020), you know more about this http.postBuffer "solution".

See commit 7a2dc95, commit 1b13e90 (22 Jan 2020) by brian m. carlson (bk2204).
(Merged by Junio C Hamano -- gitster -- in commit 53a8329, 30 Jan 2020)
(Git Mailing list discussion)

docs: mention when increasing http.postBuffer is valuable

Signed-off-by: brian m. carlson

Users in a wide variety of situations find themselves with HTTP push problems.

Oftentimes these issues are due to antivirus software, filtering proxies, or other man-in-the-middle situations; other times, they are due to simple unreliability of the network.

However, a common solution to HTTP push problems found online is to increase http.postBuffer.

This works for none of the aforementioned situations and is only useful in a small, highly restricted number of cases: essentially, when the connection does not properly support HTTP/1.1.

Document when raising this value is appropriate and what it actually does, and discourage people from using it as a general solution for push problems, since it is not effective there.

So the documentation for git config http.postBuffer now includes:

http.postBuffer

Maximum size in bytes of the buffer used by smart HTTP transports when POSTing data to the remote system.
For requests larger than this buffer size, HTTP/1.1 and Transfer-Encoding: chunked is used to avoid creating a massive pack file locally.
Default is 1 MiB, which issufficient for most requests.

Note that raising this limit is only effective for disabling chunked transfer encoding and therefore should be used only where the remote server or a proxy only supports HTTP/1.0 or is noncompliant with the HTTP standard.
Raising this is not, in general, an effective solution for most push problems, but can increase memory consumption significantly since the entire buffer is allocated even for small pushes.

17
  • 3
    This worked for me too, altho I'm a little confused as to when "smart HTTP transports" are involved in a transfer over ssh://. Mar 11, 2013 at 15:37
  • 4
    Thanks the trick worked but with double the value it was given in the answer. Aug 5, 2013 at 8:29
  • 16
    Maybe the documentation is wrong, but POST isn't what happens when you fetch/clone over HTTP. I am confused as to why the postBuffer setting has any effect in a clone or fetch. Sep 24, 2014 at 20:37
  • 2
    @Astravagrant Ok, I have updated the answer to make that value more visible.
    – VonC
    Apr 7, 2015 at 11:14
  • 3
    Beware: I experienced several issues with npm publish when raising the postBuffer. When I set it down to 50000000, issues were gone. The default value is 1000000, by the way. Oct 9, 2020 at 22:50
74

Same error with Bitbucket. Fixed by

git config --global http.postBuffer 500M
git config --global http.maxRequestBuffer 100M
git config --global core.compression 0
2
  • Tried all the solution posted here, but this one solution only worked!
    – Rich
    Jan 24, 2022 at 0:36
  • Thank you very much, worked perfectly ... Aug 29 at 16:16
19

The http.postBuffer trick did not work for me. However:

For others experiencing this problem, it may be an issue with GnuTLS. If you set Verbose mode, you may see the underlying error look something along the lines of the code below.

Unfortunately, my only solution so far is to use SSH.

I've seen a solution posted elsewhere to compile Git with OpenSSL instead of GnuTLS. There is an active bug report for the issue here.

GIT_CURL_VERBOSE=1 git clone https://github.com/django/django.git

Cloning into 'django'...
* Couldn't find host github.com in the .netrc file; using defaults
* About to connect() to github.com port 443 (#0)
*   Trying 192.30.252.131... * Connected to github.com (192.30.252.131) port 443 (#0)
* found 153 certificates in /etc/ssl/certs/ca-certificates.crt
*    server certificate verification OK
*    common name: github.com (matched)
*    server certificate expiration date OK
*    server certificate activation date OK
*    certificate public key: RSA
*    certificate version: #3
*    subject: 
*    start date: Mon, 10 Jun 2013 00:00:00 GMT
*    expire date: Wed, 02 Sep 2015 12:00:00 GMT
*    issuer: C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert High Assurance EV CA-1
*    compression: NULL
*    cipher: ARCFOUR-128
*    MAC: SHA1
> GET /django/django.git/info/refs?service=git-upload-pack HTTP/1.1
User-Agent: git/1.8.4
Host: github.com
Accept: */*
Accept-Encoding: gzip

Pragma: no-cache
< HTTP/1.1 200 OK
< Server: GitHub.com
< Date: Thu, 10 Oct 2013 03:28:14 GMT

< Content-Type: application/x-git-upload-pack-advertisement
< Transfer-Encoding: chunked
< Expires: Fri, 01 Jan 1980 00:00:00 GMT
< Pragma: no-cache
< Cache-Control: no-cache, max-age=0, must-revalidate
< Vary: Accept-Encoding
< 
* Connection #0 to host github.com left intact
* Couldn't find host github.com in the .netrc file; using defaults
* About to connect() to github.com port 443 (#0)
*   Trying 192.30.252.131... * connected
* found 153 certificates in /etc/ssl/certs/ca-certificates.crt
* SSL re-using session ID
*    server certificate verification OK
*    common name: github.com (matched)
*    server certificate expiration date OK
*    server certificate activation date OK
*    certificate public key: RSA
*    certificate version: #3
*    subject: 
*    start date: Mon, 10 Jun 2013 00:00:00 GMT
*    expire date: Wed, 02 Sep 2015 12:00:00 GMT
*    issuer: C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert High Assurance EV CA-1
*    compression: NULL
*    cipher: ARCFOUR-128
*    MAC: SHA1
> POST /django/django.git/git-upload-pack HTTP/1.1
User-Agent: git/1.8.4
Host: github.com
Accept-Encoding: gzip

Content-Type: application/x-git-upload-pack-request
Accept: application/x-git-upload-pack-result
Content-Encoding: gzip
Content-Length: 2299
* upload completely sent off: 2299out of 2299 bytes

< HTTP/1.1 200 OK
< Server: GitHub.com
< Date: Thu, 10 Oct 2013 03:28:15 GMT

< Content-Type: application/x-git-upload-pack-result
< Transfer-Encoding: chunked
< Expires: Fri, 01 Jan 1980 00:00:00 GMT
< Pragma: no-cache
< Cache-Control: no-cache, max-age=0, must-revalidate
< Vary: Accept-Encoding
< 
remote: Counting objects: 232015, done.
remote: Compressing objects: 100% (65437/65437), done.
* GnuTLS recv error (-9): A TLS packet with unexpected length was received.
* Closing connection #0
error: RPC failed; result=56, HTTP code = 200
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
4
  • 3
    i get the same verbose log like you. but solved by using larger postBuffer value.
    – suiwenfeng
    May 6, 2016 at 8:03
  • 5
    git config --global http.postBuffer 10000000000000000000000000000000
    – suiwenfeng
    May 6, 2016 at 8:03
  • 2
    Newer git versions fail due to "fatal: bad numeric config value '100000000000' for 'http.postbuffer': out of range", but setting the config value doesn't help in my case. Jan 29, 2017 at 7:53
  • The largest size I can achieve is 100000000000000
    – nhoxbypass
    Mar 17, 2020 at 6:07
18

Based on this answer, I tried following (with https url):

  1. initial cloning of repo:

git clone --depth 25 url-here

  1. fetch commits with increasing twice per try depth:

git fetch --depth 50

git fetch --depth 100

git fetch --depth 200

...and so on

  1. eventually (when I think enough is fetched) I run git fetch --unshallow - and it's done.

The process obviously takes much more time, but in my case setting http.postBuffer and core.compression didn't help.

UPD: I found out that fetching via ssh works for any repo size (discovered accidentally), done with git clone <ssh url>, given you have created ssh keys. Once repo is fetched, I change remote address using git remote set-url <https url to repo>

3
  • 1
    git clone --depth 25 url-here worked for me without using fetch Oct 1, 2020 at 11:24
  • 1
    This answer is brilliant, and was the only one which worked for me. I was using Bitbucket, and cloning smaller repos worked, while cloning larger ones did not. This fetch by chunk approach totally worked. Aug 23, 2022 at 9:10
  • 1
    Worked great for me fetching from Bitbucket. Especially since my repo was only 56 commit total anyway
    – Adam Harte
    Oct 5, 2022 at 2:54
11

The only thing that worked for me was to clone the repo using the HTTPS link instead of the SSH link.

0
11

For shared bandwidth try to clone when load is less. Otherwise, try with a high speed connection. If still does not work, please use below command,

git config --global http.postBuffer 2048M
git config --global http.maxRequestBuffer 1024M
git config --global core.compression 9

git config --global ssh.postBuffer 2048M
git config --global ssh.maxRequestBuffer 1024M

git config --global pack.windowMemory 256m 
git config --global pack.packSizeLimit 256m

And try to clone again. You might need to change those settings according to your available memory size.

10

This is due the internet connectivity issue, i faced the same issue. I did a shallow copy of code using

git clone --depth 1 //FORKLOCATION

Later unshallowed the clone using

git fetch --unshallow
2
  • This seems does the trick Sep 29, 2021 at 3:47
  • Here I am more than 6 years later - and this is the ONLY thing that seems to work. Thanks! Oct 21, 2022 at 17:13
8

Obs.: Changing http.postBuffer might also require to set up the Nginx configuration file for gitlab to accept larger body sizes for the client, by tuning the value of client_max_body_size.

However, there is a workaround if you have access to the Gitlab machine or to a machine in its network, and that is by making use of git bundle.

  1. go to your git repository on the source machine
  2. run git bundle create my-repo.bundle --all
  3. transfer (eg., with rsync) the my-repo.bundle file to the destination machine
  4. on the destination machine, run git clone my-repo.bundle
  5. git remote set-url origin "path/to/your/repo.git"
  6. git push

All the best!

0
7

If you are using https and you are getting the error.

I used https instead of http and it solved my problem

git config --global https.postBuffer 524288000
1
  • What if I'm using ssh? I cannot move to http/https. Oct 19, 2017 at 0:29
6

I got solution after using below command:

git repack -a -f -d --window=250 --depth=250

2
  • 6
    How would you run that when clone has not yet created a local git repo?
    – lucidbrot
    Sep 23, 2018 at 9:09
  • Finally been struggling with this issue, everything was not working but this command and after that git push origin main. Thanks Nov 2, 2020 at 19:27
5

I got the same issue, I fixed this with trial and error method. I changed the core.compression value until it works.

I started with "git config --global core.compression 1" after 3 attempts

"git config --global core.compression 4" worked for me.

4

Well, I wanted to push a 219 MB solution, but I had no luck with

git config --global http.postBuffer 524288000

And what's the point of having a 525 MB post buffer anyway? it's silly. So I looked at the git error below:

Total 993 (delta 230), reused 0 (delta 0)
POST git-receive-pack (5173245 bytes)
error: fatal: RPC failed; curl 56 SSL read: error:00000000:lib(0):func(0):reason(0), errno 10054

So git want's to post 5 MB, then I made the post buffer 6 MB, and it works

git config --global http.postBuffer 6291456
2
  • this does make sense. I looked at my repo size which is 15 mb. Both ssh and HTTPS complained with same error, ssh was less helpful. I have cloned larger projects without issues from github, this one was on bitbucket that just doesn't like large projects and is slow to download. Same thing happens on gitlab. Setting anything will not solve the problem. the problem here is with remote. Moving to github Setting my postbuffer close to my repo size of 15M did seem to get me through, I do not believe it is the complete solution still. Aug 21, 2016 at 5:11
  • git config --global http.postBuffer 157286400 , I set this in buffer , and changing my wifi worked.
    – ram880
    Jun 14, 2017 at 9:33
3

I was facing this issue when cloning data (via HTTP) from remote git repo hosted on AWS EC2 instance managed by elastic beanstalk. The cloning itself was also done on AWS EC2 instance.

I tried all aforementioned solutions and their combinations:

  • setting git's http.postBuffer
  • settinghttp.maxrequestbuffer
  • turning off git compression and trying "shallow" git clone and then git fetch --unshallow - see fatal: early EOF fatal: index-pack failed
  • tunning GIT memory settings - packedGitLimit et al, see here: fatal: early EOF fatal: index-pack failed
  • tunning nginx configuration - setting client_max_body_size to both big value and 0 (unlimited); setting proxy_request_buffering off;
  • setting options single-request in /etc/resolv.conf
  • throttling git client throughput with trickle
  • using strace for tracing git clone
  • considering update of git client

After all of this, I was still facing the same issue over and over again, until I found that issue is in Elastic Load Balancer (ELB) cutting the connection. After accessing the EC2 instance (the one hosting git repo) directly instead of going through ELB I've finally managed to clone git repo! I'm still not sure which of ELB (timeout) parameters is responsible for this, so I still have to do some research.

UPDATE

It seems that changing Connection Draining policy for AWS Elastic Load Balancer by raising timeout from 20 seconds to 300 seconds resolved this issue for us.

The relation between the git clone errors and "connection draining" is strange and not obvious to us. It might be that connection draining timeout change caused some internal changes in ELB configuration that fixed the issue with premature connection closing.

This is the related question on AWS forum (no answer yet): https://forums.aws.amazon.com/thread.jspa?threadID=258572

0
2

in /etc/resolv.conf add the line to the end of the file

options single-request
1
  • If the postBuffer does not help, this answer is what I suggest to try next, as that worked for me.
    – Khanh
    Aug 11, 2016 at 7:15
2

I had the same issue and it was related with a bad internet connection, so after try with some git configs, i've just disconnected from my network and connected again and it works!.

It seems that after connection lost (or the action that fires this situation), git is stuck.

I hope that it could be a help for someone more here.

Best,

2

I also had the same problem.The reason for this problem is as Kurtis's descriptions about GNUTLS.

If you have the same reason and your system is Ubuntu, you can solve this problem by installing the latest version of git from ppa:git-core/ppa.The commands are as below.

sudo add-apt-repository ppa:git-core/ppa
sudo apt-get update
sudo apt-get git
1
  • apt-get git??
    – Glenn
    Apr 22, 2020 at 19:20
2

Wasted a few hours trying some of these solutions but eventually traced this to a corporate IPS (Instrusion Protection System) dropping the connection after a certain amount of data is transferred.

2

Increasing postBuffer size and maxRequestBuffer will help you in this problem. Just follow the steps.

steps:

1 .Open terminal or Git Bash and with "cd" go to the location where you wanted to clone repo.

2.Set compression to 0

git config --global core.compression 0

3.Set postBuffer size

git config --global http.postBuffer 1048576000

4.Set maxRequestBuffer size

git config --global http.maxRequestBuffer 100M

5.Now start clone

git clone <repo url>

6.Wait till clone get complete.

Thank you. Happy Coding !!!

1

I had a similar problem, but with a bamboo job. Bamboo was failing doing a local clone (local but over an SSH proxy) of a cached repository, I deleted the cache and after that it worked, but any time it tries to clone from the local cache there is a failure. Seems like a problem with bamboo's version of the SSH proxy not git per se.

1

I faced with this problem using git in Kubuntu. I've also noticed overall instability in networking and found a solution.

in /etc/resolv.conf add the line to the end of the file

options single-request

This fixed delays before every domain name resolution and git started to work like a charm after this.

1

SOLVED WITH WIFI Router Setting :

I got same issue when I am in wifi with Settings PPPoE(auto login by wifi router).

Git download speed is very slow 15kb.

packet_write_wait: Connection to 17.121.133.16 port 22: Broken pipe fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed

Solution : 1. Changed setting to Dynamic IP, reboot wifi router. 2. From web browser login to Internet service provider portal (do not configure PPPoE , auto login from the wifi router).

After changing Git download speed is 1.7MiB.

1

use ssh instead of http, it's not a good answer for this question but at least it works for me

1

This solved my problem:

git clone --depth=20 https://repo.git -b master
1

The tricks above did not help me, as the repo was larger than the max push size allowed at github. What did work was a recommendation from https://github.com/git-lfs/git-lfs/issues/3758 which suggested pushing a bit at a time:

If your branch has a long history, you can try pushing a smaller number of commits at a time (say, 2000) with something like this:

git rev-list --reverse master | ruby -ne 'i ||= 0; i += 1; puts $_ if i % 2000 == 0' | xargs -I{} git push origin +{}:refs/heads/master

That will walk through the history of master, pushing objects 2000 at a time. (You can, of course, substitute a different branch in both places if you like.) When that's done, you should be able to push master one final time, and things should be up to date. If 2000 is too many and you hit the problem again, you can adjust the number so it's smaller.

0
1

I had to remove the branch flag for the git clone command.

1

On MacOSX High Sierra the solution for me was:

brew install git-lfs

and my repository was cloned without any errors.

0

It may be as simple as a server problem. If using GitHub, check https://twitter.com/githubstatus. I saw this for the first time just now and discovered GitHub's having a wobble. A few minutes later it worked again fine.

0

This worked for me, setting up Googles nameserver because no standard nameserver was specified, followed by restarting networking:

sudo echo "dns-nameservers 8.8.8.8" >> /etc/network/interfaces && sudo ifdown venet0:0 && sudo ifup venet0:0
0

I found my problem to be with the .netrc file, if so for you too then you can do the following:

Open your .netrc file and edit it to include github credentials. Type nano ~/netrc or gedit ~/netrc

Then include the following: *machine github.com

login username

password SECRET

machine api.github.com

login username

password SECRET*

You can include your raw password there but for security purposes, generate an auth token here github token and paste it in place of your password.

Hope this helps someone

0

I have the same error while using BitBucket. What I did was remove https from the URL of my repo and set the URL using HTTP.

git remote set-url origin http://[email protected]/mj/pt.git

Not the answer you're looking for? Browse other questions tagged or ask your own question.