32

I have been chasing a problem with extremely slow clone speeds from github on windows. Linux (kali) does not have this problem.

I am on a 100/40 megabit/s down up connection. I can obtain close to 8mb/s download no problem.

I have plugged my PC directly into our internet outlet without any router or switches.

Windows git clone (windows 10, git 2.14.1) consistently downloads at approxmately 150-210kb/s

I have tried another windows 7 and windows 10 machine with the same issue. I have tried a windows XP VM with the same issue. I have changed my connection to 4G cellular and speed is the same.

I have tried the github mirror test (https://bitbucket.org/mirror/git.git) with the same results.

I have tried SSH and HTTPS with a change in speed but still slow.

I have tried other github mirrors also with slow speeds.

I have tried using a VPN (PIA) via silicon valley which was even slower.

If I use kali linux in a VM on the same machine with the network connection bridged I get ~8 mb/s from bitbucket using both SSH and HTTPS.

If I use visual studio team services hosted build agent (windows) I get either roughly 60mb/s (possibly cached?)

I have tried running git from git bash, ubuntu bash (windows 10), windows cmd, powershell all with the same slow speeds.

I have tried the using the "OpenSSL Library" and also the "native windows secure channel library" when installing git.

I have tried downgrading as low as version 2.00 from 2.14 with no difference.

I am located in Australia.

I have spoken to bitbucket support and they have suggested that the MSS/MTU must be 1436 or lower. My router and PC defaults to 1500 however using wireshark in windows I can see that all the packets have an MTU of 1436.

Using tcpdump in kali linux I also see an MTU of 1436.

To summarise I have changed the following things: Different PC/VM Different versions of windows Different physical network connections.

I either get ~200kb/s max in windows, or I get 8mb/s max in linux.

What specifically is different between the linux and windows git implementations that could cause theses speed differences?

edit: In case git was using a slow network share (seen other questions regarding this) I tried doing a cp -r using my local clone and I get roughly 1gb/s transfer speeds (RAID 0 SSD) using the same paths as in my git config.

edit: If I bridge the network in my Kali VM to windows I get 8mb/s, if I use NAT then I get 200kb/s indicating it is something to do with the https via windows rather than the git client causing the problem. Does this help?

edit 2: It appears all HTTPS traffic via windows is capped at 200kb/s, this is a windows problem not a git problem. I will start a new question.

https://superuser.com/questions/1244551/https-traffic-40x-slower-than-http-in-windows-10-x64

15
  • Are you using https or ssh URLs when cloning directly on Windows? Can you try with simplified PATH in order to make sure no other program is interfering? (as in stackoverflow.com/a/45766037/6309)
    – VonC
    Commented Aug 19, 2017 at 7:33
  • I've tried both ssh and https like is said in the question. I don't understand the above suggestion. I've tried on clean VMs with just git and nothing else installed.
    – rollsch
    Commented Aug 19, 2017 at 11:39
  • what if you use linux client through windows as proxy, or opposite? this should detect the low-level issues like mtu size
    – max630
    Commented Aug 20, 2017 at 1:25
  • 1
    Git was written by Linus for use on linux. It uses linux filesystem and POSIX APIs. When ported to Windows/NTFS, these have to be faked out. Windows can be slower at opening many many small files, and at forking many processes. These all result in slower operation under Windows. This slowdown can be quite dramatic if you have a large repository.
    – Mort
    Commented Aug 20, 2017 at 2:10
  • 2
    Impressive. Rarely have I seen such a well thought out and investigated post. I do have one question however: is cloning something you do so often that this is going to be an issue? Even on our build servers (which start from pristine empty repo), we maintain a local cache of all external software (this is a good idea anyway, in case the internet disappears) and simply pull occasionally to update.
    – paxdiablo
    Commented Sep 2, 2017 at 2:59

3 Answers 3

16

Problem solved. Open command prompt as administrator and run the following commands:

netsh interface tcp show global
netsh interface tcp set global autotuninglevel=normal

netsh interface tcp show heuristics
netsh interface tcp set heuristics disabled

Do not forget to restart the computer in the end.

The cause was auto tuning was set to disabled for some reason on all my windows 10 PCs. After restart PC the speed instantly went to 8mb/s

2
  • 1
    Good catch. +1. I have added more details in my answer where I did suspected Windows network issue.
    – VonC
    Commented Sep 2, 2017 at 4:57
  • 1
    Perfect! This worked so good! My speed went up from 90 KB/s to 3 MB/s. Great enough! :-)
    – the_Ma5TeR
    Commented Apr 20, 2018 at 9:16
3

Try and un-compress PortableGit-2.14.1-64-bit.7z.exe in (for instance) C:\git2.14.1

Then, in a CMD (not bash), type:

set PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\
set GH=C:\git2.14.1
set PATH=%GH%\bin;%GH%\usr\bin;%GH%\mingw64\bin;%PATH%

In that session, your PATH will only reference Windows and Git.

Try again a Git clone, to see if the speed is still capped.

If it does, as the OP comment, then Windows networking is at fault.
That is what the OP discovered in his other question.
And that i what is explained in "window auto-tuning in Windows 10"

The Receive Window Auto-Tuning feature lets the operating system continually monitor routing conditions such as bandwidth, network delay, and application delay. Therefore, the operating system can configure connections by scaling the TCP receive window to maximize the network performance.

To determine the optimal receive window size, the Receive Window Auto-Tuning feature measures the products that delay bandwidth and the application retrieve rates. Then, the Receive Window Auto-Tuning feature adapts the receive window size of the ongoing transmission to take advantage of any unused bandwidth.

Let the default settings of Window Auto-Tuning feature be enabled. If your network uses an old router or your firewall software does not support this feature, and you are experiencing poor or no connectivity issues, only then may you disable this feature and see if it works in your favor.

As explained in Microsoft blog post:

Some of the confusion may have originated from a misinterpretation of a blog post which suggests disabling heuristics with the following command:

netsh interface tcp set heuristics disabled

Heuristics is a feature that can interfere with auto-tuning and disabling it can improve Internet speeds and in fact heuristics have already been disabled in Windows 8.1 and greater.
Auto-Tuning on the other hand should NEVER be disabled.

8
  • See update. It appears all HTTPS traffic is capped in windows at 200kb/s. I will start a new question.
    – rollsch
    Commented Aug 26, 2017 at 8:20
  • @rolls OK: this answer is designed to isolate Git and test if Git alone still see that effect. If you still do, then indeed, Windows (networking) itself has an issue.
    – VonC
    Commented Aug 26, 2017 at 8:23
  • @rolls Would disabinlg LSO improve the situation? (tenforums.com/network-sharing/…)
    – VonC
    Commented Aug 26, 2017 at 8:29
  • Tried git in a separate directory. Still ~190kb/s
    – rollsch
    Commented Aug 26, 2017 at 9:15
  • 1
    Don't know if it was actually enabled, but netsh interface tcp set heuristics disabled appears to have sorted out slow git clones.
    – OrangeDog
    Commented Dec 4, 2017 at 11:25
0

Probably you're just experiencing the not-so-fast performance when creating a lot of small files in an NTFS filesystem; this has nothing to do with your internet connection.

Without going too deep into technical details, NTFS is extremely inefficient when working with thousands of small files (typically what you see in a GIT repo), you can probably increase performance by tweaking the some NTFS options, anyway YMMV.

Perhaps to confirm it's a file system performance issue, you could create a RAM disk to which clone the repo.

2
  • Wasnt that. See my answer
    – rollsch
    Commented Sep 1, 2017 at 22:23
  • I also showed I could obtain 1gb/s local transfer speeds.
    – rollsch
    Commented Sep 3, 2017 at 12:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.