19

i'm trying to build a shell script to monitor some log files. I'm using a command like this:

tail -f /var/somelog | grep --line-buffered " some test and p l a c e h o l d e r" | cut -f 3,4,14 -d " "

the log file is like:

some test and p l a c e h o l d e r 3
some test and p l a c e h o l d e r 4
some test and p l a c e h o l d e r 5
some test and p l a c e h o l d e r 6

and so on.. My issue is that the output of the command does not display the last line

some test and p l a c e h o l d e r 6

until line

some test and p l a c e h o l d e r 7

is added to the log.

I hope I made clear my issue. Can anyone help me to solve this? Thank you :)

6
  • 4
    Maybe the log lines are not being ended with a line feed, but started with one instead? In this case line 6 would not really be complete until line 7 starts and provides that line feed. If you look at the log file, showing all characters, do you see a line feed at the end of the final line? Jan 16, 2013 at 14:37
  • Look for a trailing new-line ("\n") using od -c /var/somelog.
    – cdarke
    Jan 16, 2013 at 14:58
  • Do you see the same output with just tail -f ? If that is the case then @David has pointed out it correctly.
    – mtk
    Jan 16, 2013 at 16:39
  • I'm not al work at the moment, but if I run tail -f i can see the full log. I will try your solutions this evening and keep you posted :) Jan 16, 2013 at 17:30
  • 2
    my guess is that you will see the full output (including the last line) with tail -f, and even with tail -f | grep --line-buffered "pattern", but you will lose the last line once you introduce cut into the pipeline. Jan 16, 2013 at 18:34

1 Answer 1

25

the problem is almost certainly related to how grep and cut buffer their output. here's a hack that should get you around the problem, though i'm sure there are prettier ways to do it:

tail -f /var/somelog | while read line; do echo "$line" | grep "some test and p l a c e h o l d e r" | cut -f 3,4,14 -d " "; done

(don't forget the ; done at the end of the command)

alternatively, because gawk doesn't buffer it's output, you could use it in place of cut to avoid the cumbersome while loop:

tail -f log | grep --line-buffered "some test and p l a c e h o l d e r" | gawk '{print $3,$4,$14}'

check out http://www.pixelbeat.org/programming/stdio_buffering/ for more info on buffering problems.

4
  • 1
    It's also quite probable that whatever is writing to /var/somelog is buffering, so tail may not even see changes until it overflows a buffer or flushes its output some other way...
    – twalberg
    Jan 16, 2013 at 16:38
  • I have tried the first solution and is working. Thank you for your help! Jan 16, 2013 at 21:57
  • glad to help. don't forget that the best way to say thanks around here is to accept/upvote helpful answers! :) Jan 16, 2013 at 23:28
  • oh man I can't believe it but cut appears to buffer its output, whereas grep et al don't. Oh man that feels lame...
    – rogerdpack
    Jul 30, 2013 at 23:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.