This is simplest example running wget:
wget http://www.example.com/images/misc/pic.png
but how to make wget skip download if pic.png
is already available?
This is simplest example running wget:
wget http://www.example.com/images/misc/pic.png
but how to make wget skip download if pic.png
is already available?
Try the following parameter:
-nc
,--no-clobber
: skip downloads that would download to existing files.
Sample usage:
wget -nc http://example.com/pic.png
-nc
doesn't prevent the sending of the HTTP request and subsequent downloading of the file. It just doesn't do anything after downloading the file if the file has already been fully retrieved. Is there anyway to prevent making the HTTP request if the file already exists? stackoverflow.com/questions/33203898/…
– ma11hew28
Oct 18 '15 at 22:21
[ ! -e "$(basename $URL)" ] && wget $URL
– plundra
Oct 21 '15 at 11:56
--recursive
option.
– ma11hew28
Oct 22 '15 at 1:48
The -nc
, --no-clobber
option isn't the best solution as newer files will not be downloaded. One should use -N
instead which will download and overwrite the file only if the server has a newer version, so the correct answer is:
wget -N http://www.example.com/images/misc/pic.png
Then running Wget with -N, with or without
-r
or-p
, the decision as to whether or not to download a newer copy of a file depends on the local and remote timestamp and size of the file.-nc
may not be specified at the same time as-N
.
-N
,--timestamping
: Turn on time-stamping.
-N
may fail and wget will always redownload. So sometimes -nc
is better solution.
– user
Feb 23 '14 at 18:43
wget
will complain Last-modified header missing
; this is exactly the situation outlined.
– Piskvor
Feb 21 '18 at 9:58
When running Wget with -r
or -p
, but without -N
, -nd
, or -nc
, re-downloading a file will result in the new copy simply overwriting the old.
So adding -nc
will prevent this behavior, instead causing the original version to be preserved and any newer copies on the server to be ignored.
The answer I was looking for is at https://unix.stackexchange.com/a/9557/114862.
Using the
-c
flag when the local file is of greater or equal size to the server version will avoid re-downloading.
wget -i filelist.txt -c
will resume a failed download of a list of files.
– Trevor
Sep 6 '18 at 4:30
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?