28

I am using 'tail -f' to follow a log file as it's updated; next I pipe the output of that to grep to show only the lines containing a search term ("org.springframework" in this case); finally I'd like to make is piping the output from grep to a third command, 'cut':

tail -f logfile | grep org.springframework | cut -c 25-

The cut command would remove the first 25 characters of each line for me if it could get the input from grep! (It works as expected if I eliminate 'grep' from the chain.)

I'm using cygwin with bash.

Actual results: When I add the second pipe to connect to the 'cut' command, the result is that it hangs, as if it's waiting for input (in case you were wondering).

2
  • If I stop after grep (without the pipe to 'cut'), it works (without stripping the first 24 or 25 chars).
    – les2
    Jun 9, 2009 at 20:55
  • The real problem here, is that tail -f never finishes, so the rest of the pipeline keeps waiting for more input
    – Hasturkun
    Jun 11, 2009 at 0:24

3 Answers 3

30

Assuming GNU grep, add --line-buffered to your command line, eg.

tail -f logfile | grep --line-buffered org.springframework | cut -c 25-

Edit:

I see grep buffering isn't the only problem here, as cut doesn't allow linewise buffering.

you might want to try replacing it with something you can control, such as sed:

tail -f logfile | sed -u -n -e '/org\.springframework/ s/\(.\{0,25\}\).*$/\1/p'

or awk

tail -f logfile | awk '/org\.springframework/ {print substr($0, 0, 25);fflush("")}'
5
  • 1
    --line-buffered didn't work for me - that buffers the output of grep. If you do tail -f logfile|cut -c 25- it still just sits there. The buffering is being done on the output of tail. Jun 10, 2009 at 21:47
  • Revised, seems like you're being bitten by stdout buffering in cut, these should work for you
    – Hasturkun
    Jun 11, 2009 at 0:23
  • Thank you! this is really useful when pipe-chaining multiple greps. e.g. someprogwithoutput | grep --line-buffered filterpattern | grep --color highlightpattern
    – Superole
    Mar 14, 2012 at 10:35
  • Is that fflush a typo?
    – thecoshman
    Oct 19, 2015 at 13:50
  • @thecoshman: I wrote this a while ago, but I don't think it's a typo. It's a forced output flush. the "" argument is gawk specific though.
    – Hasturkun
    Oct 19, 2015 at 13:56
12

On my system, about 8K was buffered before I got any output. This sequence worked to follow the file immediately:

tail -f logfile | while read line ; do echo "$line"| grep 'org.springframework'|cut -c 25- ; done
0
-1

What you have should work fine -- that's the whole idea of pipelines. The only problem I see is that, in the version of cut I have (GNU coreutiles 6.10), you should use the syntax cut -c 25- (i.e. use a minus sign instead of a plus sign) to remove the first 24 characters.

You're also searching for different patterns in your two examples, in case that's relevant.

1
  • oops! i am using the 'minus' sign - that was a typo in my post here
    – les2
    Jun 9, 2009 at 20:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.