3

I want to create a shellscript that reads files from a .diz file, where information about various source files are stored, that are needed to compile a certain piece of software (imagemagick in this case). i am using Mac OSX Leopard 10.5 for this examples.

Basically i want to have an easy way to maintain these .diz files that hold the information for up-to-date source packages. i would just need to update these .diz files with urls, version information and file checksums.

Example line:

libpng:1.2.42:libpng-1.2.42.tar.bz2?use_mirror=biznetnetworks:http://downloads.sourceforge.net/project/libpng/00-libpng-stable/1.2.42/libpng-1.2.42.tar.bz2?use_mirror=biznetnetworks:9a5cbe9798927fdf528f3186a8840ebe

script part:

while IFS=: read app version file url md5
do 
  echo "Downloading $app Version: $version"
  curl -L -v -O $url 2>> logfile.txt
  $calculated_md5=`/sbin/md5 $file | /usr/bin/cut -f 2 -d "="`
  echo $calculated_md5    
done < "files.diz"

Actually I have more than just one question concerning this.

  1. how to calculate and compare the checksums the best? i wanted to store md5 checksums in the .diz file and compare it with string comparison with "cut"ting out the string
  2. is there a way to tell curl another filename to save to? (in my case the filename gets ugly libpng-1.2.42.tar.bz2?use_mirror=biznetnetworks)
  3. i seem to have issues with the backticks that should direct the output of the piped md5 and cut into the variable $calculated_md5. is the syntax wrong?

Thanks!

3 Answers 3

9

The following is a practical one-liner:

curl -s -L <url> | tee <destination-file> |
sha256sum -c <(echo "a748a107dd0c6146e7f8a40f9d0fde29e19b3e8234d2de7e522a1fea15048e70  -") ||
rm -f <destination-file>

wrapping it up in a function taking 3 arguments: - the url - the destination - the sha256

download() {
   curl -s -L $1 | tee $2 | sha256sum -c <(echo "$3  -") || rm -f $2
}
1
  • This saved my day! Neat use of tee and process substitution there.
    – Genzer
    Dec 20, 2021 at 6:37
6
while IFS=: read app version file url md5
do
  echo "Downloading $app Version: $version"
  #use -o for output file. define $outputfile yourself
  curl -L -v  $url -o $outputfile 2>> logfile.txt
  # use $(..) instead of backticks.
  calculated_md5=$(/sbin/md5 "$file" | /usr/bin/cut -f 2 -d "=")
  # compare md5
  case "$calculated_md5" in
    "$md5" )
      echo "md5 ok"
      echo "do something else here";;
  esac
done < "files.diz"
9
  • great! thanks for the fast answer! except the typo on line 11 (2 ";" chars) everything seems to work! i also had to change the delimiter to ";" since ":" is of course also in "http:". thanks a lot!
    – z3cko
    Jan 18, 2010 at 14:19
  • that 2 ";" chars in case/esac is valid.
    – ghostdog74
    Jan 18, 2010 at 14:22
  • the only strange thing is, that the curl -o switch seems not to work AT ALL! see it for yourself: curl -L -v -O downloads.sourceforge.net/project/libpng/00-libpng-stable/… -o libpng.tar.bz2 - any ideas?
    – z3cko
    Jan 18, 2010 at 14:25
  • strangely, bash would not let me run the script with the 2 ";" characters: syntax error near unexpected token `"do something else here"' - are you sure it is correct?
    – z3cko
    Jan 18, 2010 at 14:29
  • 1
    because you have -O switch. remove it and should be fine. check the man page to see what -O switch does.
    – ghostdog74
    Jan 18, 2010 at 14:31
0

My curl has a -o (--output) option to specify an output file. There's also a problem with your assignment to $calculated_md5. It shouldn't have the dollar sign at the front when you assign to it. I don't have /sbin/md5 here so I can't comment on that. What I do have is md5sum. If you have it too, you might consider it as an alternative. In particular, it has a --check option that works from a file listing of md5sums that might be handy for your situation. HTH.

2
  • thanks for the answer; osx does not have md5sum but md5 (which is from the openssl package)
    – z3cko
    Jan 18, 2010 at 14:18
  • furthermore, the -o switch seems not to work on osx (curl 7.16.3) - see my comment above.
    – z3cko
    Jan 18, 2010 at 14:24

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.