5

I have a text file starts with 9 digits college code and ends with of 5 digits course code.

512161000 EN5121 K. K. Jorge Institute of Engineering Education and Research, Nashik 61220 Mechanical Engineering [Second Shift] XOPENH 1 116 16978
517261123 EN5172 R. C. Rustom Institute of Technology, Shirpur 61220 Mechanical Engineering [Second Shift] YOPENH 1 100 29555
617561234 EN6175 abc xyz Education Trust, abc xyz College of Engineering,
Pune 61220 Mechanical Engineering [Second Shift] ZOPENH 2 105 25017

There are some entries where there is a line break as shown in the 3 example above. I need to merge 3rd and 4th line into one just like 1st and 2nd line, so that I can easily use command like grep, awk etc.

Update:

Kevin's answer does not seem to work.

cat todel.txt
112724510 EN1127 Jagadambha Bahuuddeshiya Gramin Vikas Sanstha's Jagdambha College of,
Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 93531

cat todel.txt | perl -ne 'chomp; if (/^\d{9}/) { print "\n$_" } else { print "$_\n" }' 
Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 93531ege of,
1
  • awk '/^[0-9]/,/[0-9]$/' # does not work
    – shantanuo
    Jun 25, 2012 at 3:30

8 Answers 8

1

Regarding split lines: This sed script assumes that you have at least one space after the leading number (on the first line of the split), and one space before the trailing number (on the last line of the split), and that there is only one split per split line.

Modified to accept input with Windows CRLF newlines or *nix LF. but note that the output is a *nix \n

sed -nr 's/\r?$// # allow for '\r\n' newlines
         /^([0-9]{9}) .* ([0-9]{5})$/{p;b}
         /^([0-9]{9}) /{h;b}
         / ([0-9]{5})$/{x;G; s/\n//; p}' 

or, shorter, but perhaps less readable:

sed -nr 's/\r?$//; /^([0-9]{9}) /{/ ([0-9]{5})$/{p;b};h;b};/ ([0-9]{5})$/{x;G; s/\n//; p}' 

I do expect that the first one is faster, because the most frequent test (for full lines) involves just a single regex, whereas the second (shorter) script, need two regex tests for the most frequent test.

This it the output I get; using GNU sed 4.2.1

512161000 EN5121 K. K. Jorge Institute of Engineering Education and Research, Nashik 61220 Mechanical Engineering [Second Shift] XOPENH 1 116 16978
517261123 EN5172 R. C. Rustom Institute of Technology, Shirpur 61220 Mechanical Engineering [Second Shift] YOPENH 1 100 29555
617561234 EN6175 abc xyz Education Trust, abc xyz College of Engineering,Pune 61220 enter code hereMechanical Engineering [Second Shift] ZOPENH 2 105 25017
112724510 EN1127 Jagadambha Bahuuddeshiya Gramin Vikas Sanstha's Jagdambha College of,Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 93531
4
  • # Does not work with the examples shown in the question. ## sed version 4.1.5
    – shantanuo
    Jun 25, 2012 at 4:51
  • That's strange; it works for me (I've added output and sed version to answer).. Does it give an error message, or just not produce the desired output? (maybe it doesn't handle the extended regex swithc -r ??)
    – Peter.O
    Jun 25, 2012 at 5:00
  • Does the file have normal *nix \n newlines (vs Windows \r\n or maybe even the old Mac \r )? I've run into problems before, when processsing Windows generated files.
    – Peter.O
    Jun 25, 2012 at 5:09
  • Good :) It's easy enough to adapt sed to hanlde the \r... I'll modify the script to handle it. Then it won't matter if you feed it \n or \r\n
    – Peter.O
    Jun 25, 2012 at 5:30
1

Assuming your data is in "file.txt", here is a scan that would put the lines back together:

cat file.txt | perl -ne 'chomp; if (/^\d{9}/) { print "\n$_" } else { print "$_\n" }'

This assumes that all valid records begin with 9-digit numbers. The "chomp" removes the new-line initially, and the pattern decides where a new-line should appear in the output.

1

This might work for you:

sed ':a;$!N;/ [0-9]\{5\}\n[0-9]\{9\} /!s/\n//;ta;P;D' file

Explanation:

  • If the line doesn't end in a space followed by five digits followed by nine digits and then a space, remove the newline.

EDIT:

Test data:

cat <<\! >/tmp/codel.txt
> 112724510 EN1127 Jagadambha Bahuuddeshiya Gramin Vikas Sanstha's Jagdambha College of,
> Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 93531
> !
sed ':a;$!N;/\s[0-9]\{5\}\n[0-9]\{9\}\s/!s/\n//;ta;P;D' /tmp/codel.txt 
112724510 EN1127 Jagadambha Bahuuddeshiya Gramin Vikas Sanstha's Jagdambha College of,Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 93531
sed ':a;$!N;/\s[0-9]\{5\}\n[0-9]\{9\}\s/!s/\n//;ta;P;D' /tmp/{codel.txt,codel.txt,codel.txt} 
112724510 EN1127 Jagadambha Bahuuddeshiya Gramin Vikas Sanstha's Jagdambha College of,Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 93531
112724510 EN1127 Jagadambha Bahuuddeshiya Gramin Vikas Sanstha's Jagdambha College of,Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 93531
112724510 EN1127 Jagadambha Bahuuddeshiya Gramin Vikas Sanstha's Jagdambha College of,Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 93531
2
  • Does not seem to work with the second example above. Question updated.
    – shantanuo
    Jun 25, 2012 at 4:15
  • @shantanuo tested using your new data (see EDIT) seems fine to me.
    – potong
    Jun 25, 2012 at 5:13
0

Maybe try to remove all line-breaks that occur after commas, like so:

perl -i -pe 's/,\n/,/g' file.txt

perhaps you want to allow for spaces after the commas:

perl -i -pe 's/(,\s*)\n/$1/g' file.txt
1
  • 1
    line break does not necessary occur after commas.
    – shantanuo
    Jun 25, 2012 at 3:24
0

try this

sed '/^[0-9]\{9\}/{h;};/^[0-9]\{9\}/!{x;G;s/\n//g;}' test | grep -E '[0-9]{5}$'
1
  • Does not work with the first example file shown in the question.
    – shantanuo
    Jun 25, 2012 at 5:08
0
awk '! ($1 ~ /^[[:digit:]]/) {$0 = save " " $0} $1 ~ /^[[:digit:]]/ {save = $0} $NF ~ /[[:digit:]]$/ {print}' inputfile
2
  • @shantanuo: The first one is the one I tested with and it works fine for me. Also, when I pipe it to wc -l I get 3. Jun 25, 2012 at 5:10
  • Awk 3.1.5 returns 4 lines instead of 3
    – shantanuo
    Jun 25, 2012 at 5:40
0
cat todel.txt |awk 'BEGIN {i=0} {first[i]=$1; lines[i++] = $0;} END {for (x=0; x<i; x++) { if ( x==(i - 1) || (first[x + 1] ~ /^[0-9]+$/ && length(first[x + 1])==9) ) {printf("%s: %s\n", x, lines[x]);} else {printf("%s: %s%s\n", x, lines[x], lines[x + 1]); x++;} } }'
0

This works with the included data set by assuming valid records end with five digits:

use Modern::Perl;

my $data = do{local $/; <DATA>};
$data =~ s/([^\d]{5})\n/$1 /sg;
say $data;


__DATA__
512161000 EN5121 K. K. Jorge Institute of Engineering Education and Research, Nashik 61220 Mechanical Engineering [Second Shift] XOPENH 1 116 16978
517261123 EN5172 R. C. Rustom Institute of Technology, Shirpur 61220 Mechanical Engineering [Second Shift] YOPENH 1 100 29555
617561234 EN6175 abc xyz Education Trust, abc xyz College of Engineering,
Pune 61220 Mechanical Engineering [Second Shift] ZOPENH 2 105 25017
112724510 EN1127 Jagadambha Bahuuddeshiya Gramin Vikas Sanstha's Jagdambha College of,
Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 93531

Output:

512161000 EN5121 K. K. Jorge Institute of Engineering Education and Research, Nashik 61220 Mechanical Engineering [Second Shift] XOPENH 1 116 16978
517261123 EN5172 R. C. Rustom Institute of Technology, Shirpur 61220 Mechanical Engineering [Second Shift] YOPENH 1 100 29555
617561234 EN6175 abc xyz Education Trust, abc xyz College of Engineering, Pune 61220 Mechanical Engineering [Second Shift] ZOPENH 2 105 25017
112724510 EN1127 Jagadambha Bahuuddeshiya Gramin Vikas Sanstha's Jagdambha College of, Engineering and Technology, Yavatmal 24510 Computer Engineering LSCO 1 55 935315
1
  • Your regex s/([^\d]{5})\n/$1 /sg; should be s/^(\d{9}.*?)(\n)[^\d{5}\n]/$1 /gm; because if the line finds a number with 5 digits in between the line and if line breaks there (which is actually the case) then your regex would fail.For ex: consider breaking at 61220 in second line.
    – user966588
    Jun 25, 2012 at 9:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.