303

I am passing a list of regex patterns to grep to check against a syslog file. They are usually matching an IP address and log entry;

grep "1\.2\.3\.4.*Has exploded" syslog.log

It's just a list of patterns like the "1\.2\.3\.4.*Has exploded" part I am passing, in a loop, so I can't pass "-v", for example.

I am confused trying to do the inverse of the above, and not match lines with a certain IP address and error so "!1.2.3.4.*Has exploded" will match syslog lines for anything other than 1.2.3.4 telling me it has exploded. I must be able to include an IP address to not match.

I have seen various similar posts on Stack Overflow. However, they use regex patterns that I can't seem to get to work with grep. What would be a working example for grep?

This is happening in a script like this;

patterns[1]="1\.2\.3\.4.*Has exploded"
patterns[2]="5\.6\.7\.8.*Has died"
patterns[3]="\!9\.10\.11\.12.*Has exploded"

for i in {1..3}
do
  grep "${patterns[$i]}" logfile.log
done
4
  • Do you mean you sometimes want to match a pattern, but other times want to match everything except a certain pattern? (this seems like an odd requirement, but whatever). In that case, why don't you iterate over two different lists of patterns?
    – beerbajay
    May 2, 2012 at 10:42
  • Well I'm not very knowledgeable about regex; I don't want to grep for "Has Exploded" because I don't want to know this about every logging device, so can I somehow grep for "Has Exploded" and !9.10.11.12 in one statement?
    – Baldrick
    May 2, 2012 at 12:06
  • If you absolutely must do it in one statement, negative lookbehinds are the way to go, as Neil suggests. See my comment there.
    – beerbajay
    May 2, 2012 at 12:57
  • Use PCRE-style regex matching, and a negative lookahead assertion, as per @Neil 's answer: patterns[3]="\!9\.10\.11\.12.*Has exploded" changes to patterns[3]="(?<!9\.10\.11\.12).*Has exploded" and grep "${patterns[$i]}" logfile.log changes to grep -P "${patterns[$i]}" logfile.log PCRE assumes more metacharacters by default, so some of the escapes may need to be removed from other matching expressions.
    – Codex24
    Jun 5, 2019 at 16:45

4 Answers 4

553

grep matches, grep -v does the inverse. If you need to "match A but not B" you usually use pipes:

grep "${PATT}" file | grep -v "${NOTPATT}"
10
  • This is going into the middle of a loop as I mentioned and I'm just passing the PATTERN to grep so I can't use "-v" as I mentioned. I'm just looping round a list of PATTERNs and passing to grep.
    – Baldrick
    May 2, 2012 at 10:21
  • 3
    You can indeed use -v and you can use it in a loop. Perhaps you need to be more specific about your limitations, or perhaps you have a misconception about how your script should work. Try posting some code.
    – beerbajay
    May 2, 2012 at 10:25
  • Thanks beerbajay, I have added a code snipped to the original post to give some context. Do you see what I mean now?
    – Baldrick
    May 2, 2012 at 10:36
  • This answer isn't completely correct but you were pretty much write beerbajay, I needed to rethink the loop and in use -v in the end. Thanks for the pointer ;)
    – Baldrick
    May 3, 2012 at 16:05
  • 1
    But what if A is composed of B? In other words, what if I want to match lines with no A and lines with AB ? A pipe will not work.
    – pawamoy
    Dec 2, 2015 at 20:25
21
(?<!1\.2\.3\.4).*Has exploded

You need to run this with -P to have negative lookbehind (Perl regular expression), so the command is:

grep -P '(?<!1\.2\.3\.4).*Has exploded' test.log

Try this. It uses negative lookbehind to ignore the line if it is preceded by 1.2.3.4.

5
  • 3
    I'm pretty sure that grep doesn't support lookaround. Unless you're using Gnu grep and use the --P parameter to make it use a PCRE engine. May 2, 2012 at 10:15
  • No, grep doesn't support this type of Regex; $grep -P (?<\!1\.2\.3\.4) test.log -bash: syntax error near unexpected token `('
    – Baldrick
    May 2, 2012 at 10:20
  • You're going to need to quote the regex if it contains characters which would be interpreted by the shell.
    – beerbajay
    May 2, 2012 at 10:26
  • correct quoting: grep -P '(?<!1\.2\.3\.4) Has exploded' test.log Note that the lookbehind only works on the characters immediately preceding the matching part of the expression, so if there's other things between the address and message, e.g. 1.2.3.4 FOO Has exploded, this won't work.
    – beerbajay
    May 2, 2012 at 12:57
  • @TimPietzcker, very observant. I'll add that to the question. Also, please note that there is a .* after the negative lookbehind since his example also has it, I imagine there might be other text in between.
    – Neil
    May 2, 2012 at 13:01
13

It seems no one has posted a blend of the best of all answers, regex (-E) with match inversion (-v)

grep -Ev 'pattern1|pattern2|pattern3' file

Notably, no lookarounds required, so this works if your grep version doesn't have -P available.

2
patterns[1]="1\.2\.3\.4.*Has exploded"
patterns[2]="5\.6\.7\.8.*Has died"
patterns[3]="\!9\.10\.11\.12.*Has exploded"

for i in {1..3}
  do
grep "${patterns[$i]}" logfile.log
done

should be the the same as

egrep "(1\.2\.3\.4.*Has exploded|5\.6\.7\.8.*Has died)" logfile.log | egrep -v "9\.10\.11\.12.*Has exploded"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.