1

I'm trying to look in a variable passed to my program (the variable is $1) and to replace any special characters with quoted forms of said special character, in order to not have the special character actually do what it normally would.

My code is

#!/bin/sh
target="$1"
newtarget=`echo "$target" | sed -e s/\*/\\*/g`
newtarget=`echo "$newtarget" | sed -e s/\^/\\^/g`
newtarget=`echo "$newtarget" | sed -e s/\+/\\+/g`
newtarget=`echo "$newtarget" | sed -e s/\-/\\-/g`
newtarget=`echo "$newtarget" | sed -e s/\\/\\\/g`
newtarget=`echo "$newtarget" | sed -e s/\./\\./g`
newtarget=`echo "$newtarget" | sed -e s/\$/\\$/g`
newtarget=`echo "$newtarget" | sed -e s/\[/\\[/g`
newtarget=`echo "$newtarget" | sed -e s/\]/\\]/g`
sed s/"$newtarget"/"$2"/g "$3" > "$3.updated"
mv "$3.updated" $3

My first line, with $target, should look in the target string and see if there's a * in the string. If there is, it will replace it with *. The reason that in the code, it appears as * and then \*, is so that the program doesn't see * and think it wants to actually use *, it just sees * as a regular character by quoting it with . I've done the same thing in all of the other lines, but with different characters. After the first one, it should check in newtarget and do the same thing, but with different characters.

What my overall program should do, is that it's passed 3 parameters, the first is a string to be replaced, the second is a string to replace it, and the third is a filename. So by the end of it, if the file was originally something like

aa\^a*aa$aa[aaa$a]a 

And I supply

"a\^a*" "test"

as parameters, the result should be

atestaa$aa[aaa$a]a 

But my code still doesn't work. What's wrong with my code? I don't know if my sed syntax is right with coding, or if my additional statements don't work, or if I have to special quoting for some of the special characters.

EDIT: I know that I should be able to do this with multiple sed commands like I have, but I don't know why they aren't working right, so I'm pretty sure it's something to do with my quoting in the actual sed commands at the end of the "newtarget=" lines.

EDIT2: I have quoted my sed arguments in my code now, but it's still not working properly. Are there special ways I need to quote certain special characters? I assumed that putting a backslash in front of each character would properly quote it.

#!/bin/sh
target="$1"
newtarget=`echo "$target" | sed -e 's/\*/\\*/g'`
newtarget=`echo "$newtarget" | sed -e 's/\^/\\^/g'`
newtarget=`echo "$newtarget" | sed -e 's/\+/\\+/g'`
newtarget=`echo "$newtarget" | sed -e 's/\-/\\-/g'`
newtarget=`echo "$newtarget" | sed -e 's/\\/\\\/g'`
newtarget=`echo "$newtarget" | sed -e 's/\./\\./g'`
newtarget=`echo "$newtarget" | sed -e 's/\$/\\$/g'`
newtarget=`echo "$newtarget" | sed -e 's/\[/\\[/g'`
newtarget=`echo "$newtarget" | sed -e 's/\]/\\]/g'`
sed s/"$newtarget"/"$2"/g "$3" > "$3.updated"
mv "$3.updated" $3
2
  • Quote the argument to sed -e. By the way, you can collapse them into one, too: sed =e 's/\*/\\&/g' -e 's/\^/\\&/g' … or even sed -e 's/[][*^+\\.$-]/\\&/g'. You will also need to escape the slash (delimiter).
    – mirabilos
    Dec 6, 2013 at 15:21
  • I tried the last bit of code you gave me, but I wasn't able to get it to work, so I went back to my original code and just quoted the sed -e argument like you said, but I'm still getting an error. When you say I need to escape the slash delimiter, do you mean in all my lines, or just the one where I have a lot of them ('s/\\/\\\/g')
    – JC2112
    Dec 6, 2013 at 15:34

2 Answers 2

4

The purpose of the multiple invocations of sed is to place a literal backsplash before each occurrence of a set of characters. This can be done in one call to sed, but you need to be careful about how you specify the set.

First, let's see what the general command will look like:

newtarget=$( echo "$target" | sed -e 's/\([...]\)/\\\1/g'

where ... will be replaced with the set of characters to escape. This commands uses parentheses to capture a single instance of one of those characters, the replaces it with a backsplash followed by the captured character. To specify the set of characters, use

[]*^+\.$[-]

Two notes: first, the ] must come first so that it isn't mistaken for the end of the set, since [] is an invalid set. Second, - must come last, so that it isn't mistaken as the range operator (e.g., [a-z] is the set of lowercase letters, but [az-] is simply the three characters a, z, and -).

Putting it all together:

 newtarget=$( echo "$target" | sed -e 's/\([]*^+\.$[-]\)/\\\1/g' )
10
  • I tried implementing this as newtarget=echo "$target" | sed -e 's/(]*^+\.$[-)/\\\1/g'`` but I get an error of an unterminated `s' command, as well as "no previous regular expression". I assumed you meant the [...] was just meant to symbolize that I should put the set there, not that i should put it inside brackets (because then we'd have a [] right at the beginning if I put ] first.
    – JC2112
    Dec 6, 2013 at 15:58
  • No, the surrounding [...] are necessary, as the specify a regular expression that matches one of the characters they contain. See my comment about the empty set [] being invalid, so placing ] first allows it to be interpreted literally, instead of as the closing bracket of the set.
    – chepner
    Dec 6, 2013 at 16:01
  • Hmm, alright. If I wanted to include another special character in the set, how would I do that? For instance, if I wanted to include ` , would I just pop it in between + and ^, or does it require something special like the brackets and -?
    – JC2112
    Dec 6, 2013 at 16:05
  • My command looks like newtarget=echo "$target" | sed -e 's/\([]*^+\.$[-]\)/\\\1/g' now, but I'm getting the error "Invalid back reference". Are there any special requirements I should be aware of regarding any of the characters? Also, there are ` before the e in echo and after the comma after g, but the comment is funky with formatting
    – JC2112
    Dec 6, 2013 at 16:08
  • I think different versions of sed have different rules about what is and isn't escaped. Try removing the backslashes before the parentheses to see if that helps.
    – chepner
    Dec 6, 2013 at 16:15
2

The problem with what you've done is that you haven't quoted your sed expressions. For example, write

sed s/\*/\\*/

either as

sed 's/\*/\\*/'

or

sed s/\*/\\\\*/

I'm not sure why do you need that elaborate function in order to escape special characters. You could define a function that would give back the escaped input string:

myescape() { printf "%q" "$1"; }

%q

causes printf to output the corresponding argument in a format that can be reused as shell input.

Another function for passing arguments to sed:

myreplace() { sed "s/$1/$2/" <<< "$3"; }

Now you could invoke it by saying:

myreplace "$(myescape 'pattern')" "replacement" "original_string"

Example:

$ myescape() { printf "%q" "$1"; }
$ myreplace() { sed "s/$1/$2/" <<< "$3"; }
$ myreplace $(myescape 'a\^a*') 'test' 'aa\^a*aa[aaa]a'
atestaa[aaa]a
8
  • That's not the way we're supposed to do it, however. I'm supposed to do it in this manner with sed commands, and I want to understand why my code isn't working.
    – JC2112
    Dec 6, 2013 at 15:35
  • @JoJoBya You should clearly state in your question that this is homework and that you aren't looking for a solution to your problem but instead want people to fix your code.
    – devnull
    Dec 6, 2013 at 15:37
  • That printf %q is GNU bash-specific anyway, and he needs the sed special characters escaped, not the shell special characters.
    – mirabilos
    Dec 6, 2013 at 15:37
  • @mirabilos If you're jumping in, you might very well figure that the problem in OPs code is the shell itself.
    – devnull
    Dec 6, 2013 at 15:40
  • @JoJoBya The problem with your script is that you haven't quoted the sed expression. There are additional problems with specific ones.
    – devnull
    Dec 6, 2013 at 15:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.