70

I'm trying to remove leading and trailing space in 2nd column of the below input.txt:

Name, Order  
Trim, working
cat,cat1

I have used the below awk to remove leading and trailing space in 2nd column but it is not working. What am I missing?

awk -F, '{$2=$2};1' input.txt

This gives the output as:

Name, Order  
Trim, working
cat,cat1

Leading and trailing spaces are not removed.

8 Answers 8

101

If you want to trim all spaces, only in lines that have a comma, and use awk, then the following will work for you:

awk -F, '/,/{gsub(/ /, "", $0); print} ' input.txt

If you only want to remove spaces in the second column, change the expression to

awk -F, '/,/{gsub(/ /, "", $2); print$1","$2} ' input.txt

Note that gsub substitutes the character in // with the second expression, in the variable that is the third parameter - and does so in-place - in other words, when it's done, the $0 (or $2) has been modified.

Full explanation:

-F,            use comma as field separator 
               (so the thing before the first comma is $1, etc)
/,/            operate only on lines with a comma 
               (this means empty lines are skipped)
gsub(a,b,c)    match the regular expression a, replace it with b, 
               and do all this with the contents of c
print$1","$2   print the contents of field 1, a comma, then field 2
input.txt      use input.txt as the source of lines to process

EDIT I want to point out that @BMW's solution is better, as it actually trims only leading and trailing spaces with two successive gsub commands. Whilst giving credit I will give an explanation of how it works.

gsub(/^[ \t]+/,"",$2);    - starting at the beginning (^) replace all (+ = zero or more, greedy)
                             consecutive tabs and spaces with an empty string
gsub(/[ \t]+$/,"",$2)}    - do the same, but now for all space up to the end of string ($)
1                         - ="true". Shorthand for "use default action", which is print $0
                          - that is, print the entire (modified) line
11
  • 1
    Can you please explain this.
    – Marjer
    Dec 15, 2013 at 23:28
  • 3
    Look at @EdMorton's answer for a single gsub solution. It also use the character class for space which is a better thing to do. Dec 7, 2017 at 21:46
  • Ed Morton's answer is here. Feb 4, 2019 at 20:16
  • used the regex.. gsub(/^[ \t]+/,"",$2) May 7, 2019 at 8:58
  • 2
    This isn't the right answer, because it removes all spaces, not just leading and trailing.
    – Peter
    Mar 26, 2020 at 7:19
39

remove leading and trailing white space in 2nd column

awk 'BEGIN{FS=OFS=","}{gsub(/^[ \t]+/,"",$2);gsub(/[ \t]+$/,"",$2)}1' input.txt

another way by one gsub:

awk 'BEGIN{FS=OFS=","} {gsub(/^[ \t]+|[ \t]+$/, "", $2)}1' infile
8
  • I like how you are removing only leading and trailing spaces by using the ^ and $ anchors in two separate gsub commands. More complex, but on the money.
    – Floris
    Dec 16, 2013 at 14:05
  • 1
    I just think it directly, and thanks for detail explanation for Ganz Ricanz
    – BMW
    Dec 16, 2013 at 23:23
  • 2
    Feel free to improve your answer by writing your explanation of what you did (or copy mine). I think your answer should be accepted - but it is always better when it is "complete".
    – Floris
    Dec 16, 2013 at 23:44
  • This is the only answer that works properly, so it should be the accepted answer.
    – Geoff
    Jul 26, 2020 at 5:10
  • 1
    yes, and the command will be simpler. awk '$1=$1' infile > outfile
    – BMW
    Aug 13, 2020 at 21:36
22

Warning by @Geoff: see my note below, only one of the suggestions in this answer works (though on both columns).

I would use sed:

sed 's/, /,/' input.txt

This will remove on leading space after the , . Output:

Name,Order
Trim,working
cat,cat1

More general might be the following, it will remove possibly multiple spaces and/or tabs after the ,:

sed 's/,[ \t]\?/,/g' input.txt

It will also work with more than two columns because of the global modifier /g


@Floris asked in discussion for a solution that removes trailing and and ending whitespaces in each colum (even the first and last) while not removing white spaces in the middle of a column:

sed 's/[ \t]\?,[ \t]\?/,/g; s/^[ \t]\+//g; s/[ \t]\+$//g' input.txt

*EDIT by @Geoff, I've appended the input file name to this one, and now it only removes all leading & trailing spaces (though from both columns). The other suggestions within this answer don't work. But try: " Multiple spaces , and 2 spaces before here " *


IMO sed is the optimal tool for this job. However, here comes a solution with awk because you've asked for that:

awk -F', ' '{printf "%s,%s\n", $1, $2}' input.txt

Another simple solution that comes in mind to remove all whitespaces is tr -d:

cat input.txt | tr -d ' '
11
  • I don't think your current awk solution removes trailing spaces…?
    – Floris
    Dec 16, 2013 at 0:08
  • While the main point in my answer is to show that sed is better suited for this job than awk - because it's a text editing task, the awk script should work. I've tested it. What is wrong with that?
    – hek2mgl
    Dec 16, 2013 at 0:10
  • Your sed command doesn't appear to remove trailing spaces either. Just put > output.txt at the end, open the result in a text editor and you will see...
    – Floris
    Dec 16, 2013 at 0:12
  • There are no trailing spaces before the first column. The task is more: Remove the spaces after the ,
    – hek2mgl
    Dec 16, 2013 at 0:13
  • trailing spaces at the end of the second column. Quoting from the question "I'm trying to remove leading and trailing space in 2nd column "
    – Floris
    Dec 16, 2013 at 0:14
22

I just came across this. The correct answer is:

awk 'BEGIN{FS=OFS=","} {gsub(/^[[:space:]]+|[[:space:]]+$/,"",$2)} 1'
0
5

just use a regex as a separator:

', *' - for leading spaces

' *,' - for trailing spaces

for both leading and trailing:

awk -F' *,? *' '{print $1","$2}' input.txt
1
  • Fails to remove the trailing spaces from colum 2. Instead removes the trailing spaces from column 1.
    – Geoff
    Jul 26, 2020 at 4:59
2

Simplest solution is probably to use tr

$ cat -A input
^I    Name, ^IOrder  $
  Trim, working  $
cat,cat1^I  

$ tr -d '[:blank:]' < input | cat -A
Name,Order$
Trim,working$
cat,cat1
2
  • 1
    Nice and compact. Can you modify this so it only trims the second column (per the question)?
    – Floris
    Dec 16, 2013 at 0:15
  • Fails since it removes all spaces, from both columns, and appends spurious '$'.
    – Geoff
    Jul 26, 2020 at 5:02
2

The following seems to work:

awk -F',[[:blank:]]*' '{$2=$2}1' OFS="," input.txt
1
  • Fails and adds spurious commas
    – Geoff
    Jul 26, 2020 at 5:00
-1

If it is safe to assume only one set of spaces in column two (which is the original example):

awk '{print $1$2}' /tmp/input.txt

Adding another field, e.g. awk '{print $1$2$3}' /tmp/input.txt will catch two sets of spaces (up to three words in column two), and won't break if there are fewer.

If you have an indeterminate (large) number of space delimited words, I'd use one of the previous suggestions, otherwise this solution is the easiest you'll find using awk.

2
  • Fails badly, and I won't even try to explain, but try " Multiple spaces , and here ".
    – Geoff
    Jul 26, 2020 at 5:04
  • Again, this particular solution assumes only ONE set of spaces between TWO columns as SPECIFIED by the actual problem. If you want to pretend a complex gsub regex replacement is 'easier' than a simple awk print statement, I won't even try to explain why your're wrong.
    – Andrew
    Aug 2, 2020 at 17:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.