I have the following variable.
echo "|$COMMAND|"
which returns
|
REBOOT|
How can I remove that first newline?
The tr
command could be replaced by ${parameter/pattern/string}
bashism:
COMMAND=$'\nREBOOT\r \n'
echo "|${COMMAND}|"
|
OOT
|
echo "|${COMMAND//[$'\t\r\n']}|"
|REBOOT |
echo "|${COMMAND//[$'\t\r\n ']}|"
|REBOOT|
See Parameter Expansion and QUOTING in bash's man page:
man -Pless\ +/parameter/pattern/string bash
man -Pless\ +/\/pattern bash
man -Pless\ +/\\\'string\\\' bash
man -Pless\ +/^\\\ *Parameter\\\ Exp bash
man -Pless\ +/^\\\ *QUOTING bash
${parameter//pattern/string} If there are two slashes separating parameter and pattern, all matches of pattern are replaced with string.
As asked by @AlexJordan, this will suppress all specified characters. So what if $COMMAND
do contain spaces...
COMMAND=$' \n RE BOOT \r \n'
echo "|$COMMAND|"
|
BOOT
|
read -r COMMAND <<<"${COMMAND//[$'\t\r\n']}"
echo "|$COMMAND|"
|RE BOOT|
Answering Vulwsztyn's question:
Why does this work when the pattern is empty?
In ${COMMAND//[$'\t\r\n ']}
:
/
mean: Pattern substitutionpattern
is /[$'\r\n ']
, begin with /
then all matches of pattern are replaced with stringstring
is empty.tr
for single string!Let compare:
COMMAND=$'\nREBOOT\r \n'
echo ${COMMAND@Q}
$'\nREBOOT\r \n'
COMMAND=$(echo $COMMAND|tr -d '\n\t\r ')
echo ${COMMAND@Q}
'REBOOT'
Then
time for i in {1..1000};do
COMMAND=$'\nREBOOT\r \n'
COMMAND=$(echo $COMMAND|tr -d '\n\t\r ')
done;echo ${COMMAND@Q}
real 0m2.785s
user 0m2.296s
sys 0m0.774s
'REBOOT'
With
COMMAND=$'\nREBOOT\r \n'
COMMAND="${COMMAND//[$'\t\r\n ']}"
echo ${COMMAND@Q}
time for i in {1..1000};do
COMMAND=$'\nREBOOT\r \n'
COMMAND="${COMMAND//[$'\t\r\n ']}"
done;echo ${COMMAND@Q}
real 0m0.006s
user 0m0.001s
sys 0m0.004s
'REBOOT'
Doing 1'000 forks to tr
take more than 2700ms on my host, while same job is done in 6ms ( 464.2x faster!! ) by using built-in bash Parameter Expansion!!
COMMAND="RE BOOT"; echo "|${COMMAND//[$'\t\r\n ']}|"
returns |REBOOT|
Mar 16, 2016 at 11:48
\n
to prevent this: COMMAND="RE BOOT"; echo "|${COMMAND//[$'\t\r\n']}|"
will return |RE BOOT|
.
Mar 16, 2016 at 22:01
${COMMAND//[$'\t\r\n']}
? I thought ${COMMAND//[\t\r\n]}
would simply work, but it didn't. What is the $
symbol for and the single quotes too?
Clean your variable by removing all the linefeeds:
COMMAND=$(echo $COMMAND|tr -d '\n')
tr
. Simply COMMAND=$(echo $COMMAND)
will give a similar effect. Which is, presumably, devil spawn as it invokes a new process, but it's still pretty short and sweet for the human's eyes and if you have a second or two to spare in your life you may be willing to take the hit :-) .
${COMMAND//[$'\t\r\n']}
Nov 10, 2021 at 16:36
+1
for noticing the antipattern echo $COMMAND
(missing quotes). But it's an antipattern and shouldn't be used since it will also expand glob characters if there are any (e.g., *
, or [...]
or ?
). So never use an unquoted expansion, unless your variable is in fact a glob that you want to actually expand.
Jan 6 at 10:18
echo "|$COMMAND|"|tr '\n' ' '
will replace the newline (in POSIX/Unix it's not a carriage return) with a space.
To be honest I would think about switching away from bash to something more sane though. Or avoiding generating this malformed data in the first place.
Hmmm, this seems like it could be a horrible security hole as well, depending on where the data is coming from.
tr -d '\n'
for dropping instead of replace by a space
Oct 13, 2013 at 16:20
tr '\n' ' ' <<< "|$COMMAND|"
instead of echo ... | ...
Oct 13, 2013 at 16:24
Using bash
:
echo "|${COMMAND/$'\n'}|"
(Note that the control character in this question is a 'newline' (\n
), not a carriage return (\r
); the latter would have output REBOOT|
on a single line.)
Uses the Bash Shell Parameter Expansion ${parameter/pattern/string}
:
The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string. [...] If string is null, matches of pattern are deleted and the / following pattern may be omitted.
Also uses the $''
ANSI-C quoting construct to specify a newline as $'\n'
. Using a newline directly would work as well, though less pretty:
echo "|${COMMAND/
}|"
#!/bin/bash
COMMAND="$'\n'REBOOT"
echo "|${COMMAND/$'\n'}|"
# Outputs |REBOOT|
Or, using newlines:
#!/bin/bash
COMMAND="
REBOOT"
echo "|${COMMAND/
}|"
# Outputs |REBOOT|
Adding answer to show example of stripping multiple characters including \r using tr and using sed. And illustrating using hexdump.
In my case I had found that a command ending with awk print of the last item |awk '{print $2}'
in the line included a carriage-return \r as well as quotes.
I used sed 's/["\n\r]//g'
to strip both the carriage-return and quotes.
I could also have used tr -d '"\r\n'
.
Interesting to note sed -z
is needed if one wishes to remove \n line-feed chars.
$ COMMAND=$'\n"REBOOT"\r \n'
$ echo "$COMMAND" |hexdump -C
00000000 0a 22 52 45 42 4f 4f 54 22 0d 20 20 20 0a 0a |."REBOOT". ..|
$ echo "$COMMAND" |tr -d '"\r\n' |hexdump -C
00000000 52 45 42 4f 4f 54 20 20 20 |REBOOT |
$ echo "$COMMAND" |sed 's/["\n\r]//g' |hexdump -C
00000000 0a 52 45 42 4f 4f 54 20 20 20 0a 0a |.REBOOT ..|
$ echo "$COMMAND" |sed -z 's/["\n\r]//g' |hexdump -C
00000000 52 45 42 4f 4f 54 20 20 20 |REBOOT |
And this is relevant: What are carriage return, linefeed, and form feed?
What worked for me was echo $testVar | tr "\n" " "
Where testVar contained my variable/script-output
If you are using bash with the extglob option enabled, you can remove just the trailing whitespace via:
shopt -s extglob
COMMAND=$'\nRE BOOT\r \n'
echo "|${COMMAND%%*([$'\t\r\n '])}|"
This outputs:
|
RE BOOT|
Or replace %% with ## to replace just the leading whitespace.
You can simply use echo -n "|$COMMAND|"
.
$ man echo
-n
do not output the trailing newline
Use this bashism if you don't want to spawn processes like (tr, sed or awk) for such a simple task. Bash can do that alone:
COMMAND=${COMMAND//$'\n'/}
From the documentation:
${FOO//from/to} Replace all
${FOO/from/to} Replace first match
CLEANED=${COMMAND//[$'\t\r\n']}
. I didn't downvote because of its effectiveness in writing bash scripts, I downvoted because it shows up before F. Hauri's answer, which is more complete and includes pointers to the man pages and so on. It adds nothing new.
To address one possible root of the actual issue, there is a chance you are sourcing a crlf file.
CRLF Example:
.env (crlf)
VARIABLE_A="abc"
VARIABLE_B="def"
run.sh
#!/bin/bash
source .env
echo "$VARIABLE_A"
echo "$VARIABLE_B"
echo "$VARIABLE_A $VARIABLE_B"
Returns:
abc
def
def
If however you convert to LF:
.env (lf)
VARIABLE_A="abc"
VARIABLE_B="def"
run.sh
#!/bin/bash
source .env
echo "$VARIABLE_A"
echo "$VARIABLE_B"
echo "$VARIABLE_A $VARIABLE_B"
Returns:
abc
def
abc def
Note that the shell already does that for you if you pass $COMMAND
as a parameter instead of a string.
So you can try this:
COMMAND="a
b
c"
echo $COMMAND
To add the pipes, you can just use the quotes and no spaces like so:
echo "|"$COMMAND"|"
works just fine and it's a tad simpler than the other solutions. It also works with just /bin/sh
.
Grep can be used to filter out any blank lines. This is useful if you have several lines of data and want to remove any empty.
echo "$COMMAND" | grep .