0

I don't quite understand how to solve my problem even though I have been using grep, uniq and sort before. I would very much appreciate how to crack this up :)

I wanted to get the uniq of the first 6 characters from my input file and get the output as shown below. I don't know if it's uniq, grep, awk I need to use, maybe someone could give me a hand in this.

My file looks like this:

Field1     Filed2    Field3
value1   some_stuff  something
value2   another     fake  
value1   fake        value    
value3   blah        blah
value2   blah        fake 


Prefered output:

Field1    Field2    Field3
value1   some_stuff something
value2   another    fake
value3   blah       blah
3
  • 1
    It is always advisable to add your efforts along with samples too in your post @rockStar Nov 23, 2018 at 10:50
  • 1
    Possible duplicate of Remove line if field is duplicate
    – kvantour
    Nov 23, 2018 at 11:17
  • sorry guys, will do add it next time...quit bz doing stuff atm :/
    – rockStar
    Nov 23, 2018 at 11:43

1 Answer 1

2

Could you please try following,

awk 'FNR==1{print;next} !a[substr($0,1,6)]++' Input_file

Explanation: Adding explanation for above code.

awk '
FNR==1{                     ##Checking condition if line is first then do following.
  print                     ##Printing current line which is first line of headers.
  next                      ##next will skip all further lines from here.
}                           ##Closing condition BLOCK here.
!a[substr($0,1,6)]++        ##Creating array named a whose index is first 6 characters and keeping its increment value.
                            ##awk works on function condition/pattern and action, no action mentioned here so print of line happened.
'  Input_file               ##Mentioning Input_file name here.

In case your first field has only 6 characters then use following.

awk '!a[$1]++' Input_file

About the !a[$1]++ part. Basically it checks if it had already stored in an array (named x here) the first column value on a previous line parsing.

If it is the case (a[$1] != 0), it doesn't output the line. Otherwise, it outputs it and stores it (a[$1]++, so a[$1] = a[$1] + 1 therefore a[$1] will equal 1) for the next lines parsing. See this Unix answer.

1
  • 1
    thanx this worked and also for the explaination, im saving this oneliner as i would be using it more often :)
    – rockStar
    Nov 23, 2018 at 11:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.