1

I'm trying to populate some exclusions to run a backup script on the command line by reading the values from an external file and prefixing these with -x. There will be a variable number of exclusions, including none at all. The trouble I'm running into is that when bash expands this variable, it frames it with single quotes. This confuses the backup script.

#cat sss2.sh:

#!/bin/bash
IFS=$'\n'
xcl=""
for x in $(cat /root/exfile)
 do
 xcl="$xcl -x $x"
 done
backupbin /sourcepath /destination "$xcl"

# cat exfile
"*.tgz"
"*.zip"

When I run this, bash expands the variable $xcl with single quotes, which prevents backupbin from running. It is due to the spaces, because if I didn't have the -x, the single quotes disappear.

# /bin/bash -x ./sss2
+ IFS=''
+ xcl=
++ cat exfile
+ for x in '$(cat /root/exfile)'
+ xcl=' -x "*.tgz"'
+ for x in '$(cat exfile)'
+ backupbin /sourcepath /destination ' -x "*.tgz" -x "*.zip"'

I have tried this as expanding a populated array, and have tried different combinations of single quotes, double quotes, and back ticks. I've tried using eval without success:

backupbin /sourcepath /destination $(eval echo "$xcl")

Finally, I've tried creating an array and expanding this, but I get the same result:

IFS=$'\n' 
excludefile=($(cat /root/exfile))
if [ ${#excludefile[@]} -eq 0 ]; then
 echo ""
 else
 for i in "${excludefile[@]}"
 do
 printf "%s" " -x $i"
 done

What am I doing wrong?

5
  • Give a try using backslash where you are using spaces.
    – smali
    Jun 4, 2014 at 4:42
  • or go through this stackoverflow.com/questions/836334/…
    – smali
    Jun 4, 2014 at 4:44
  • @basin He doesn't need to use eval if he wants the shell to expand them. With $(cat /root/exfile), glob patterns would still be expanded. Word splitting is not the only thing that happens there. The behaviour should also be the same with $(</root/exfile).
    – konsolebox
    Jun 4, 2014 at 6:03
  • That's the actual reason why for x in $<something> is to be avoided unless pathname expansion is really intended and relying from IFS would be safe enough. There's a safer solution to it actually, but I'm not going to allow another pioneer idea to be stolen by posturing FAQ/tutorial authors who actually just gathered ideas from old forums :)
    – konsolebox
    Jun 4, 2014 at 6:11
  • Thanks, I didn't know that. Do you know how to expand this? glob='* *'
    – basin
    Jun 4, 2014 at 6:14

2 Answers 2

5

As always with this sort of problem, the simplest solution to this kind of problem is to use an array, rather than to try to juggle quotes. (Quote+eval solutions almost always fail in some corner case, sometimes disastrously.)

#!/bin/bash
IFS=$'\n'

xcl=()
for x in $(< /root/exfile); do
  xcl+=(-x "$x")
done

backupbin /sourcepath /destination "${xcl[@]}"

See the bash FAQ.

Edit: as @konsolebox notes, the use of $(</root/exfile) is subject to pathname expansion. So if the file contained a line like "foo*" and one or more files whose names start with foo exist in the current working directory, then the exclusion pattern -- which was probably intended to be a wildcard exclusion -- will instead by replaced by the list of filenames. It would better to use:

#!/bin/bash
xcl=()
while IFS= read -r x; do xcl+=(-x "$x"); done < /root/exfile
backupbin /sourcepath /destination "${xcl[@]}"
2

@rici's suggestion about using an array is already correct. As for reading lines, better use this form:

while read -r x; do
    xcl+=(-x "$x")
done < /root/exfile

Reading values through word splitting is not recommended.

4
  • 1
    Good point about word splitting, but unfortunately you still need to set IFS to get read to work correctly in the case that a line begins or ends with whitespace: while IFS= read -r x; will work. mapfile is another possibility.
    – rici
    Jun 4, 2014 at 6:50
  • This will definitely not expand globs.
    – basin
    Jun 4, 2014 at 7:43
  • @basin Yep I made the answer before making the comment and seeing the possibility that the OP would actually want them expanded.
    – konsolebox
    Jun 4, 2014 at 8:11
  • @rici It's rare for filenames to be having leading and trailing spaces but yes that could be helpful as well. About mapfile yes I thought about it (readarray) but you have to insert -x before every element anyway that you'd still run a loop so I just decided to skip it.
    – konsolebox
    Jun 4, 2014 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.