53

I have a file with three columns. I would like to delete the 3rd column(in-place editing). How can I do this with awk or sed?

123   abc  22.3
453   abg  56.7
1236  hjg  2.3

Desired output

123  abc
453  abg
1236 hjg 
1

8 Answers 8

71

try this short thing:

awk '!($3="")' file
7
  • 52
    This doesn't actually delete the given column; it sets it to the empty string, but you still get an extra FS in your output. This may or not be important, depending on what you're doing with the transformed data.
    – larsks
    Mar 17, 2014 at 14:04
  • try this to save the generate output to a newfile. awk '!($3="")' file > newfile Apr 9, 2015 at 20:05
  • 3
    @A.Danischewski this is not good way to go, what happens if the awk script has error? you lost your file. take this awk '..' file > tmp && mv tmp file
    – Kent
    Jul 7, 2016 at 8:24
  • @A.Danischewski there are other circumstances that can affect: what if the file system has problems of space?
    – fedorqui
    Jul 8, 2016 at 11:41
  • 3
    That will also recompile the current record replacing all white space between fields with single blank characters and removing any leading and/or trailing white space. It will not produce the OPs desired output given his posted input. To do that you need stackoverflow.com/a/38145415/1745001.
    – Ed Morton
    Feb 15, 2017 at 21:07
43
+100

With GNU awk for inplace editing, \s/\S, and gensub() to delete

1) the FIRST field:

awk -i inplace '{sub(/^\S+\s*/,"")}1' file

or

awk -i inplace '{$0=gensub(/^\S+\s*/,"",1)}1' file

2) the LAST field:

awk -i inplace '{sub(/\s*\S+$/,"")}1' file

or

awk -i inplace '{$0=gensub(/\s*\S+$/,"",1)}1' file

3) the Nth field where N=3:

awk -i inplace '{$0=gensub(/\s*\S+/,"",3)}1' file

Without GNU awk you need a match()+substr() combo or multiple sub()s + vars to remove a middle field. See also Print all but the first three columns.

3
  • 2
    Note: on Ubuntu Trusty GNU Awk 4.0.1 doesn't have the awk inplace extension enabled by default.
    – user4401178
    Jul 7, 2016 at 1:41
  • i think you mean /\s+\S+/ with an s+ and not /\s*\S+/ Jan 4, 2022 at 2:02
  • @BrianWiley no because then it wouldn't work for the first field if there were no preceding spaces.
    – Ed Morton
    Jan 4, 2022 at 2:39
25

This might work for you (GNU sed):

sed -i -r 's/\S+//3' file

If you want to delete the white space before the 3rd field:

sed -i -r 's/(\s+)?\S+//3' file
6
  • 3
    @potong, is \S means all characters that are not a space ? Where it is documented ? Mar 12, 2013 at 13:37
  • 1
    What's -r do? My sed doesn't have it. Feb 18, 2015 at 12:55
  • 3
    @JoshuaCheek -r is the GNU sed-specific option to enable EREs (google that). If you use -E instead of -r it'll work in GNU sed plus some other seds.
    – Ed Morton
    Jul 3, 2016 at 12:29
  • 1
    @GillesQuenot Yes, exactly. You can see it documented in the GNU manual in the section on regular expression extensions. May 22, 2018 at 14:33
  • this only works for the first row for my GNU sed Jan 4, 2022 at 1:40
18

It seems you could simply go with

awk '{print $1 " " $2}' file

This prints the two first fields of each line in your input file, separated with a space.

1
  • 3
    This assumes only 3 columns. You'll otherwise need a loop: awk '{printf $1 OFS $2; for(i=4;i<=NF;i++) printf OFS $i; printf ORS}' file (OFS defaults to a space and ORS defaults to a newline).
    – Adam Katz
    Jul 6, 2016 at 0:57
17

Try using cut... its fast and easy

First you have repeated spaces, you can squeeze those down to a single space between columns if thats what you want with tr -s ' '

If each column already has just one delimiter between it, you can use cut -d ' ' -f-2 to print fields (columns) <= 2.

for example if your data is in a file input.txt you can do one of the following:

cat input.txt | tr -s ' ' | cut -d ' ' -f-2

Or if you better reason about this problem by removing the 3rd column you can write the following

cat input.txt | tr -s ' ' | cut -d ' ' --complement -f3

cut is pretty powerful, you can also extract ranges of bytes, or characters, in addition to columns

excerpt from the man page on the syntax of how to specify the list range

Each LIST is made up of one range, or many ranges separated by commas.
Selected input is written in the same order that it is read, and is
written exactly once. Each range is one of:

  N     N'th byte, character or field, counted from 1
  N-    from N'th byte, character or field, to end of line
  N-M   from N'th to M'th (included) byte, character or field
  -M    from first to M'th (included) byte, character or field

so you also could have said you want specific columns 1 and 2 with...

cat input.txt | tr -s ' ' | cut -d ' ' -f1,2
3
  • 2
    I know this isn't an answer to what has been asked, but it sure is the best answer!
    – Yan Foto
    Nov 7, 2021 at 22:02
  • 1
    agreed, the rest of the answers deal with white space which will not work if you have spaces in a column for a file that is tab delimited. this will work for tab delimited cut -f1-241,243-267,269-278 -d$'\t' which would remove columns 242 and 268. Jan 4, 2022 at 2:21
  • 1
    This is the actual correct way to do it. It can also easily delete multiple columns simultaneously, which the sed solution is not able to do.
    – Tianyi Shi
    Aug 2, 2022 at 7:34
7

Try this :

awk '$3="";1' file.txt > new_file && mv new_file file.txt

or

awk '{$3="";print}' file.txt > new_file && mv new_file file.txt
1

Try

awk '{$3=""; print $0}'
-1

If you're open to a Perl solution...

perl -ane 'print "$F[0] $F[1]\n"' file

These command-line options are used:

  • -n loop around every line of the input file, do not automatically print every line

  • -a autosplit mode – split input lines into the @F array. Defaults to splitting on whitespace

  • -e execute the following perl code

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.