28

My script

export IFS=":"

cat hello | while read a b c d; do
    echo $a,$b,$c,$d
done

My file hello

1:1:1:1
2:2:2:2
3:3:3:3

My output

1,1,1,1
2,2,2,2

If I put a blank line after 3:3:3:3 in hello then the output becomes

1,1,1,1
2,2,2,2
3,3,3,3

Anyone know how I can fix this problem so I don't need to put a blank line at the end of hello?

0

5 Answers 5

27

What's happening is, the read command fails when the input is not terminated with a newline. Since the newline character is missing at the end of your file, the read fails, so the last iteration of the while loop is skipped.

If you don't want to / cannot make sure that your input file has a newline at the end, you can group your cat with an echo to give the appearance of an input terminated by newline, for example like this:

{ cat hello; echo; } | while read a b c d; do
    echo $a,$b,$c,$d
done

or like this:

(cat hello; echo) | while read a b c d; do
    echo $a,$b,$c,$d
done
0
12

This is an extensive reading for the question why?

If you don't want to put a new line at the end, you can do this:

while IFS=: read -r a b c d || [ -n "$a" ]; 
do     
echo $a,$b,$c,$d; 
done < file

Or using grep:

while IFS=: read -r a b c d; 
do     
echo $a,$b,$c,$d; 
done < <(grep "" filee)

Note:

  1. There's no need to use cat with while loop.
  2. You should almost always use the -r option with read.
1
  • 1
    This worked best for me thank you. @janos' answer works too, but uses a subshell - anything set inside the while loop in their answer lives only in that scope (e.g. if you're exporting variables from a file to use outside of the while loop - you want to use this answer)
    – ctwheels
    Commented Jan 6, 2021 at 17:24
9

There exists a not very well-known, nasty hack to append a missing final newline to a text stream if and only if it's missing:

sed '$a\'

See this discussion on unix.stackexchange, for example.

1
  • 4
    There are lots of those. Others include grep ^, awk 1, paste
    – rici
    Commented Nov 15, 2013 at 22:09
1

It is most likely due to absence of newline character after last line in your input file.

Run this command to find out:

cat -vte hello

And see what you get.

1

Your file is probably missing a newline (\n) on the last line. You can prove this by printing the residual values of $a etc, like so

while IFS=: read a b c d; 
do     
echo $a,$b,$c,$d; 
done < hello  
echo $a,$b,$c,$d

Fix the file. Also, you should be able to set IFS specifically for the read as shown above. And no need to cat the file either, just use input redirection as shown

Not the answer you're looking for? Browse other questions tagged or ask your own question.