-1

Roughly how many TB should be needed to hold a PyPI mirror for Linux only, using bandersnatch 3.6.0 installed with anaconda, and the default bandersnatch.conf plus:

[plugins]
enabled =
    exclude_platform

[blacklist]
platforms =
    windows
    macos
    freebsd

?

Spoiler alert: I'm up to 3.9 TB already (!) :)

Thank you very much.

-- Doug

P.S. I guess this comes down to: How much of PyPI is specific to Windows, macOS, and FreeBSD, as opposed to being general, or Linux-specific? https://pypi.org/stats/ says: "Sum of release files -- ALL of PyPI: 6.6 TB"

4
  • I don't believe there're FreeBSD packages at PyPI. There're source distributions and binary wheels for Linux, MacOS and w32. And only for AMD/Intel processors, no ARMs or MIPS etc. For ARMs there is piwheels.org
    – phd
    May 9 '20 at 17:36
  • 1
    As for the question — PyPI is most probably the only place where you can ask this question.
    – phd
    May 9 '20 at 17:39
  • Where exactly do PyPI people discuss these kinds of things? I looked but couldn't find a likely place to ask the question. But I'd now go there and put a link to the answer, if anyone can point me in the right direction. Thanks.
    – Doug
    May 11 '20 at 18:07
  • I was hoping that someone from the PyPI cognoscenti would answer with a search / query of some sort that would let me ask how many Linux and platform-general packages there are. Then I could just multiply by 3KB -- the average size of the first 10,000 wheels that were downloaded. But no such luck :(
    – Doug
    May 12 '20 at 2:13
0

bandersnatch mirror download finally finished: 4,148 GB used.

(Took 230 hours, over a residential internet connection that Ookla https://www.speedtest.net/ reports as anywhere between 25 and 75 Mbps depending on time of day and how many Zoom sessions / youtube videos the residents are running.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.