1

I have the following log file that I'd like to parse in C#.

I've gone down the route of using a RegEx to get most of it split. I've tested this in RegExr with MultiLine (m) flag checked.

Log

5376:0084 2015-08-07 13:51:29.103 Error ### Error Message ###
5376:0084 2015-08-07 13:51:35.545 Error Discarding invalid session
    System.Web.Services.Protocols.SoapException: Verify Session ID failed
    at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
    at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
5376:0084 2015-08-07 13:51:36.013 Error ### Error Message ###

Split to Table:

| ProcessID | DateTime                | Type  | Message               |
|-----------|-------------------------|-------|-----------------------|
| 5376:0084 | 2015-08-07 13:51:29.103 | Error | ### Error Message ### |

I've used the following pattern

string pattern = @"(.*:\d{4}) ((\d{4})-(\d{2})-(\d{2}) (\d{2}):(\d{2}):(\d{2}).(\d{3})) ([A-Za-z\n]+) (.*$)";

This gets lines 1,3 & 6 but I'd like to gather lines 2-5 into one group. So "Discarding ... parameters)" would be the whole Message.

1
  • Please explain what difference you need in the question itself. Commented Aug 18, 2015 at 9:13

4 Answers 4

1

Note that in log parsing named captures are a great help, I strongly advise to use them. Also, you can have more control over what you capture with . using an inline singleline modifier (?s:...). This way, you do not have to use a global RegexOptions.Singleline option, and you still can use . to match any symbol but a newline.

Here is my attempt:

var pattern = @"(?m)^(?<ProcessID>\d{4}:\d{4})\s+(?<DTime>\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\.\d+)\s+(?<Type>\w+)\s+(?<Message>(?s:.*?(?=\n\d+:\d+|\r?\z)))";

Here, (?m) sets the multiline mode for ^ to match a line start, then I modified the ID and datetime subpatterns for more efficient ones with \d{n}, the Type part can be actually adjusted to your needs (e.g. [\w\s]+), and the Message part will only match arbitrary number of characters up to the XXX:XXXX on a new line (due to \n\d+:\d+) or up to the end of the string (\z). See regex demo, see Table tab.

enter image description here

1
  • Thanks, I love the idea 'named captures', this will give my code more meaning Commented Aug 18, 2015 at 9:22
1

You need to match also the newline characters which exists at the Message part. This could be achieved by using DOTALL modifier s.

@"(?s)(\d+:\d{4}) (\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{3}) ([A-Za-z\n]+) (.*?)(?=\n\d+:|$)"

or

@"(?s)(?:\n|^)(\d+:\d{4}) (\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{3}) ([A-Za-z\n]+) (.*?)(?=\n\d+:|$)"

DEMO

6
  • Thanks, can I just remove Groups 1-10 from Group 0 to get the difference? Commented Aug 18, 2015 at 9:07
  • ya, see the demo link. Four capturing groups would be enough. Commented Aug 18, 2015 at 9:10
  • There are only 5 groups, indexed from 0 to 4 in a match result with this regex. You cannot remove 10 groups then. Commented Aug 18, 2015 at 9:11
  • ya, i excluded the 0th group(which contains all the matched chars). Commented Aug 18, 2015 at 9:11
  • @AvinashRaj: Nice joke. 0th group cannot be "excluded", if it were, it would mean a no match. :) Also, there is a singleline flag/modifier, not DOTALL in C#. Commented Aug 18, 2015 at 9:15
0

Alex. Try this:

string pattern = @"^(\S+) (\S+ \S+) (\S+) ((?:.*(?:\n\s)?)+)";

(Sample is here: https://regex101.com/r/uI4uQ0/1)

  1. (\S+) = "5376:0084";
  2. (\S+ \S+) = "2015-08-07 13:51:29.103"
  3. (\S+) = "Error" (one word only)
  4. ((?:.*(?:\n\s)?)+) = "...session
        System...
        at..."

Magick is here: "\n\s". It says that we need line-break AND any whitespace char.

Good luck, MiKe.

0

Besides regex, for parsing log file, you have a possibility to use TextFieldParser class. Although it is an unnecessary dependance on this assebmly Microsoft.VisualBasic.FileIO.TextFieldParser, but it is a good one.

Here is a good tutorial on how to consume this class.

3
  • This won't help since one of the fields contains newline characters. Commented Aug 18, 2015 at 9:16
  • @stribizhev, perhaps using Environment.NewLine in the same way as vbTab were used here: reader.Delimiters = New String() {"," , vbTab} Commented Aug 18, 2015 at 9:19
  • From my experience, I could not manage to correctly handle entries with newlines using TextFieldParser, I had to perform a thorough pre-process. Commented Aug 18, 2015 at 9:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.