14

I have a file that I am attempting to strip customer names from using AWK. The file is a fixed width file, and every column has meaning.

The file consists of many lines, all the same format, very similar to the below:

1234-123   123456 12345678901234CUSTOMER NAME TO REMOVE12345-1234 TRN   123-123   12345678901-1234  TRN 12345678        
1234-123   123456 12345678901234CUSTOMER NAME TO REMOVE12345-1234 TRN   123-123   12345678901-1234  TRN 12345678        
1234-123   123456 12345678901234CUSTOMER NAME TO REMOVE12345-1234 TRN   123-123   12345678901-1234  TRN 12345678        
1234-123   123456 12345678901234CUSTOMER NAME TO REMOVE12345-1234 TRN   123-123   12345678901-1234  TRN 12345678

It is the customer name that I need to swap with an imaginary name so that the desired output is:

1234-123   123456 12345678901234SENTINAL PRIME         12345-1234 TRN   123-123   12345678901-1234  TRN 12345678        
1234-123   123456 12345678901234OPTIMUS PRIME          12345-1234 TRN   123-123   12345678901-1234  TRN 12345678        
1234-123   123456 12345678901234BUMBLE BEE             12345-1234 TRN   123-123   12345678901-1234  TRN 12345678        
1234-123   123456 12345678901234IRON HIDE              12345-1234 TRN   123-123   12345678901-1234  TRN 12345678

I have a list of transformer names that I would like to use for this, stored in a file called transformer.names.

SENTINEL PRIME
OPTIMUS PRIME
BUMBLEBEE
IRONHIDE

However, to keep each line of the original file the same width, I need to right pad the transformer names with spaces as the transformer names I have are all different lengths.

It seems to be possible to right pad these names to a certain length using AWK but I have not managed to figure it out (or find a clear enough answer) for me to understand yet.

Below is my current AWK script.

#!/usr/bin/awk -f
BEGIN {
}
{
  getline line < "transformer.names"
  print substr($0, 0, 30) line substr($0, 62, 120)
}

I run it with this command:

my_program.awk my-file.txt

I think I can include a line something like this in place of the print line above, however I have not managed to get it working yet.

printf "-%32s|", substr($0, 0, 30) line substr($0, 62, 120)

Any tips would be fantastic!

2
  • Could you please more sample for input will be helpful for us to help you then. Commented Jun 22, 2018 at 2:23
  • Done. Cheers :) Commented Jun 22, 2018 at 2:38

6 Answers 6

9

You need to apply the %Ns to the specific field you want to pad not the whole line and you need to make the minus (for leftpad/rightalign) part of the specifier, and also printf does not automatically add a line/record separator as print does so you need to add that:

 printf "%s%-32s%s\n", substr($0, 1, 30), newname, substr($0, 62, 120)
 # note commas; this is a format string containing three specifiers, 
 # and separate three data values used for those three specifiers

Alternatively you could pad the field and then concatenate:

 print substr($0,1,30) sprintf("%-32s", newname) substr($0,62,120) 
 # no commas except within the sprintf (and the substr's) 

If your data file has more lines than your 'transformernames' file, then you need to buffer the names and cycle through them repeatedly, as Ravinder shows.

Also, substr positions in awk start at 1; if you specify 0 or negative, it is treated as 1 but I think it's clearer to actually say what you mean, so I fixed that. 62 is not the correct starting position for the part after the customer name in the example data you posted, but you said that data is only 'very similar' to the real data, so I don't know whether 56 or 62 or something else is correct.

2

Could you please try following and let me know if this helps you. So it will have all transformer names and let's say it has lesser values than Input_file lines then it will keep printing lines from starting of it.

awk '
FNR==NR{
  a[FNR]=$0;
  count=FNR;
  next}
{
  val=val==count?1:++val;
  print substr($0,1,32) a[val]"\t\t"substr($0,56)
}' transformer.names  Input_file

Explanation: Adding explanation for above code too now.

awk '
FNR==NR{                                          ##Checking condition here FNR==NR which will be TRUE when first Input_file is being read.
  a[FNR]=$0;                                      ##Creating an array named a whose index is FNR and value is current line.
  count=FNR;                                      ##Creating variable count whose value is FNR value(current line number value of first Input_file).
  next}                                           ##next will skip further statements from here onward.
{                                                 ##This block will execute when 2nd Input_file is being read.
  val=val==count?1:++val;                         ##Creating variable val whose value is increment each time and when it is equal to count it is set to 1 then.
  print substr($0,1,32) a[val]"\t\t"substr($0,56) ##Printing sub-string from 1 to 32 chars, value of a[val] TABs then sub-string from 56 char to till last of line.
}' transformer.names  Input_file                  ##Mentioning Input_file(s) name here.
2
  • 1
    Thank you, I certainly will! Are you able to describe how the key parts of the code work? It's not intuitive for someone just coming into this. Commented Jun 22, 2018 at 3:57
  • Tabs will only work on some output devices and when the field ending column is a multiple of 8 and the substituted values don't vary in length by more than 7. Commented Jun 22, 2018 at 4:19
2
#!/usr/bin/awk -f
BEGIN {
}
{
  getline line < "transformer.names"
  printf("%s %-32s %s \n", substr($0, 0, 30), line, substr($0, 62, 120))
}

You pretty much had the answer in your question! I just copied yours and modified it a little :)

2

Your data seems not to be Uppercase letters before the text you want to modify.
So you can try this awk.

awk '
FNR==NR {
  a[NR]=$0
  b=length()
  len = len < b ? b : len
  next
}
{
  c = sprintf( "%-*2$s" , a[FNR], (len+1))
  sub(/[A-Z][A-Z ]+/,c)
}
1' transformer_name customer_name

First we get all the transformer name into array a and keep the bigger length into len After we replace all old name by new name adjusting the formatting in c.
You can modify (len+1) to your need.

1
  • Sorry - that was misleading. I fixed it in the question. Commented Jun 28, 2018 at 23:20
0

or just use FS and skip the substr() calls :

mawk 'NF *=(__=FNR)==NR ? (___[__]=$_)*_ : (OFS=sprintf("%-24s",___[__]))^_'\
                                             FS='CUSTOMER NAME TO REMOVE' 

      <( echo 'SENTINEL PRIME
               OPTIMUS PRIME
               BUMBLEBEE
                IRONHIDE') -

1234-123   123456 12345678901234SENTINEL PRIME          12345-1234 TRN   123-123   12345678901-1234  TRN 12345678        
1234-123   123456 12345678901234OPTIMUS PRIME           12345-1234 TRN   123-123   12345678901-1234  TRN 12345678        
1234-123   123456 12345678901234BUMBLEBEE               12345-1234 TRN   123-123   12345678901-1234  TRN 12345678        
1234-123   123456 12345678901234IRONHIDE                12345-1234 TRN   123-123   12345678901-1234  TRN 12345678
0

Here is a solution that uses column to do all the right-padding math for you -- very handy if your input data suddenly changes length, but does not satisfy the OP's requirement that the output line-lengths match the original line-lengths:

cat my-file.txt | \
awk '{getline line < "transformer.names"; \
print substr($0, 0, 30) line "|" substr($0, 62, 120)}' | \
column -s'|' -t

And a combination of gsub and column could be used without any need to grab substrings (or know their length).

cat my-file.txt | \
awk -v pipe='|' \
'{getline line < "transformer.names"; \
gsub("CUSTOMER NAME TO REMOVE",line pipe,$0); \
print $0}' | \
column -s'|' -t

Here ^ we create a variable called pipe within awk, place it inside of the substitution gsub( ... line pipe ....) and then use that as a delimeter for column later on. The use of the pipe is arbitrary. We could do the exact same thing with, say, an asterisk (star):

cat my-file.txt | \
awk -v star='*' \
'{getline line < "transformer.names"; \
gsub("CUSTOMER NAME TO REMOVE",line star,$0); \
print $0}' | \
column -s'*' -t

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.