How to download a file with wget using multiple connection in which each connection downloads a part of the file?
3 Answers
use aria2
aria2c -x 16 [url] #where 16 is the number of connections
OR
Just repeat the wget -r -np -N [url]
for as many threads as you need. This isn’t pretty and there are surely better ways to do this, but if you want something quick and dirty it should do the trick.
Please look at http://blog.netflowdevelopments.com/2011/01/24/multi-threaded-downloading-with-wget/
-
6The solution you gave for wget downloads a single file multiple times resulting in multiple copies, but I want to download a single file using multiple connection, each of which downloads its part of file. Mar 1, 2014 at 13:18
-
1
wget
solution doesn't download one file in multiple threads: The options used-r
is recursive,-np
(--no-parent
) — don't ascend to the parent directory,-N
(--timestamping
) — don't re-retrieve files unless newer than local. Butwget
would definitely work if you're downloading a mirror of a site. Apr 7, 2015 at 14:44 -
@Trix -x, --max-connection-per-server=NUM The maximum number of connections to one server for each download. Possible Values: 1-16 Default: 1 Tags: #basic, #http, #ftp Feb 9, 2016 at 14:38
-
-
1
sudo apt-get install axel
axel -n 5 url
Would do the work!
Axel is a lightweight program that helps with the downloads and supports multiple connections.
Want to resume download after a disconnection? No problem, just repeat the command and axel will take care of it.
-
Not very useful with long S3 pre-signed URLs due to
Can't handle URLs of length over 1024
limitation Sep 29, 2021 at 11:40