0

I am trying to process a file

File: /var/log/audit/audit.log:
        /var/paas/sys/log/kubernetes/audit.log
        /var/paas/sys/log/csms-storagemgr/audit.log
File: /var/log/sudo.log: Good
File: /var/log/secure: Good

For lines not started with 'File:', I need to combine it with above line

Expected result will be

File: /var/log/audit/audit.log /var/paas/sys/log/kubernetes/audit.log /var/paas/sys/log/csms-storagemgr/audit.log
File: /var/log/sudo.log: Good
File: /var/log/secure: Good

How to achieve this goal with sed or something other tools for it?

I have tried with

sed -ne '/File:/{:a;N;/File:/!{ba};s/\n/\t/g;p}' /tmp/test

but it failed.

3 Answers 3

2

This might work for you (GNU sed):

sed -E '/^File:/{:a;N;/\nFile:/!s/\n\s*/ /;ta;P;D}' file

Match a line beginning File:.

Append the next line.

If the appended line does not begin File:, replace the newline and any other following white space by a single space.

If the substitution was successful, loop back and append the next line etc.

Otherwise, print/delete the first line in the pattern space and begin the sed cycle again.

N.B. Trying to read beyond the end of the file with the N command, will do what you would expect and print the current line.

1
  • Um, failed. If there are 2 parts in the file need to be combined. It will only combined the first part. For my example, if the first 3 lines was duplicated again, inserted from line 4. It will fail to combine.
    – matrixzj
    Nov 17, 2020 at 19:03
2

With GNU awk, assuming amount of spaces do not matter.

$ awk -v RS='File:' 'NF{$1=$1; print RS, $0}' ip.txt
File: /var/log/audit/audit.log: /var/paas/sys/log/kubernetes/audit.log /var/paas/sys/log/csms-storagemgr/audit.log
File: /var/log/sudo.log: Good
File: /var/log/secure: Good
  • RS='File:' sets File: as input record separator
  • NF so that empty records are not printed
  • $1=$1 rebuild input record based on output field separator, which is space by default
  • print RS, $0 print value of RS followed by space and then value of rebuilt input record

Default input field separator is single space character, which will remove leading/trailing space/tab/newline characters and splits a record into fields based on one or more continuous sequence of those three characters. Here's another example:

$ printf '    a  \t  b      3   '  | awk '{$1=$1; print}'
a b 3
2
  • Thanks a lot, it works. But I have a confusion with $1=$1, for example: echo 'col1,col2' | awk 'BEGIN{FS=",";OFS="|"}{$1=$1;print}' will work as expected with result col1|col2. But echo 'col1,col2' | awk 'BEGIN{FS=",";OFS="|"}{print}' will not. But checking input stream as echo 'col1,col2' | awk 'BEGIN{FS=",";OFS="|"}{print $1}', col1 will be shown as result. That means that input stream has been split as expected.
    – matrixzj
    Nov 17, 2020 at 18:50
  • @matrixzj yeah, awk will rebuild the input record with new value of OFS only if NF or some field value is changed
    – Sundeep
    Nov 18, 2020 at 1:25
0

Try this maybe?

awk '/^File:/ && f { printf "\n" }
  { printf "%s", $0; f=1 }
  END { if (f) printf "\n" }' /tmp/test

This omits the newline from the end of each line, then adds it back when it sees the next File: line. Notice the END block to add back the very final newline after we are done with all the input lines, too.

Demo: https://ideone.com/SCfkx2

If you want to normalize the spaces from beginning of line, try adding gsub(/^[ \t]+/, " ") before the printf "%s", $0.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.