5

I have .sh script that, among other things, wgets currency exchange rates from Google as follows:

printf 'Bash: Going to get exchange rates'
echo    
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=aud" |  sed '/res/!d;s/<[^>]*>//g' > exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=jpy" |  sed '/res/!d;s/<[^>]*>//g' >> exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=hkd" |  sed '/res/!d;s/<[^>]*>//g' >> exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=nzd" |  sed '/res/!d;s/<[^>]*>//g' >> exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=eur" |  sed '/res/!d;s/<[^>]*>//g' >> exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=gbp" |  sed '/res/!d;s/<[^>]*>//g' >> exrates

mv /home/stan/perl/2014/scripts/exrates /home/stan/perl/2014/exrates/exrates

printf 'Bash: Got exchange rates'
echo

Occasionally script hangs here, however. I don't mind not updating these rates every time it runs, if it hangs I'd like to skip this step altogether, but how?

What should I put in "if" statement to check if wget can fetch data promptly or will take forever? A little more verbosity in wget execution wouldn't hurt either.

Btw, I don't know why wget hangs. Browser opens those pages okay, and same commands run from terminal line by line work, too.

3 Answers 3

4

I assume that it hangs because you have a number of HTTP requests being sent to a single host in a script. The host in question doesn't like that too much and it starts to block requests from your IP address.

A simple workaround would be to put a sleep in between the requests. You could also make use of a function:

getExchangeRates() {
  wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=$1" |  sed '/res/!d;s/<[^>]*>//g' >> exrates
  sleep 10   # Adding a 10 second sleep
}

and invoke it by passing a parameter to the function:

getExchangeRates aud

The function could also be invoked in a loop for various currencies:

for currency in aud jpy hkd nzd eur gpb; do
  getExchangeRates $currency
done
3
  • Somehow invoking this function doesn't work, no parameters get passed or recognized. Commented Apr 22, 2014 at 11:40
  • @Stangoesagain There was a typo in the parameter being passed to the function. Fixed now.
    – devnull
    Commented Apr 22, 2014 at 11:43
  • @devnull Got it, it's either "for currency .. do getExchangeRates $currency" or "for i .. do getExchangeRates $i". Working now. Big thanks Commented Apr 22, 2014 at 11:48
4

use timeout in wget statement only

wget --timeout 10 <URL>

timeout is in seconds and put some sleep in between two wgets

2

wget has various timeout options. From the man page

   --timeout=seconds
       Set the network timeout to seconds seconds.  This is equivalent to
       specifying --dns-timeout, --connect-timeout, and --read-timeout,
       all at the same time.

So you can simply set --timeout, or if you believe it's one of the other factors alone you can set a specific timeout

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.