2

I try to use regular expression to capture data group from below logs. The pattern is

<item> : <key> = <value> , <key> = <value>, ..., <key> = <value>

([#\w\d]*?)[\s]*=[\s]*([.\w\d]*) can capture group <key> and group <value> but i want to capture the <item> group as well, so i group the above and repeat using {n}.

([\w]*):([\s]*(([#\w\d]*?)[\s]*=[\s]*([.\w\d]*)),*){1,}

20141207,07:15:52,0,>>RATIO: casher#=30, Value=2.579,Units=ratio,Error=N 20141207,07:15:52,0,>>RATIO: casher#=31, Value=4.509,Units=ratio,Error=N 20141207,07:15:52,0,>>RATIO: casher#=32, Value=3.735,Units=ratio,Error=N 20141207,07:15:52,0,>>RATIO: casher#=33, Value=2.401,Units=ratio,Error=N

20141207,07:15:52,0,>>CUSTOMER: casher#=30, Value=50,Units= count 20141207,07:15:52,0,>>CUSTOMER: casher#=31, Value=6,Units= count 20141207,07:15:52,0,>>CUSTOMER: casher#=32, Value=88,Units= count 20141207,07:15:52,0,>>CUSTOMER: casher#=33, Value=33,Units= count

pic1 enter image description here

obviously the result is not what is expecting. can anybody give me some tips? i am using python eventually to translate to code. thank you.

1
  • I'm not sure what the question is, but you can't capture all key=value pairs with a single regex. Not in separate groups, anyway.
    – Aran-Fey
    Commented Jan 13, 2015 at 9:33

2 Answers 2

3
(?<=>>)(\w+):|([\w#]+)\s*=\s*(\S+?)(?:,|\s)

Try this.Grab the capture.See demo.

https://regex101.com/r/fA6wE2/1

NODE                     EXPLANATION
--------------------------------------------------------------------------------
  (?<=                     look behind to see if there is:
--------------------------------------------------------------------------------
    >>                       '>>'
--------------------------------------------------------------------------------
  )                        end of look-behind
--------------------------------------------------------------------------------
  (                        group and capture to \1:
--------------------------------------------------------------------------------
    \w+                      word characters (a-z, A-Z, 0-9, _) (1 or
                         more times (matching the most amount
                         possible))
--------------------------------------------------------------------------------
   )                        end of \1
--------------------------------------------------------------------------------
  :                        ':'
--------------------------------------------------------------------------------
  |                        OR
 --------------------------------------------------------------------------------
  (                        group and capture to \2:
--------------------------------------------------------------------------------
    [\w#]+                   any character of: word characters (a-z,
                         A-Z, 0-9, _), '#' (1 or more times
                         (matching the most amount possible))
--------------------------------------------------------------------------------
  )                        end of \2
--------------------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                       more times (matching the most amount
                       possible))
--------------------------------------------------------------------------------
  =                        '='
--------------------------------------------------------------------------------
  \s*                      whitespace (\n, \r, \t, \f, and " ") (0 or
                       more times (matching the most amount
                       possible))
--------------------------------------------------------------------------------
  (                        group and capture to \3:
--------------------------------------------------------------------------------
    \S+?                     non-whitespace (all but \n, \r, \t, \f,
                         and " ") (1 or more times (matching the
                         least amount possible))
--------------------------------------------------------------------------------
  )                         end of \3
--------------------------------------------------------------------------------
  (?:                      group, but do not capture:
--------------------------------------------------------------------------------
    ,                        ','
--------------------------------------------------------------------------------
   |                        OR
--------------------------------------------------------------------------------
    \s                       whitespace (\n, \r, \t, \f, and " ")
--------------------------------------------------------------------------------
  )                        end of grouping
2
  • 1
    i love you @vks, i am weak, would you please explain a little bit on the expression? thank you very much. i accepted the ans. thank you very much.
    – pinky
    Commented Jan 13, 2015 at 9:44
  • thank you for the explanation and teaching. your matching pattern also matched key=value even if the "item :" not exist, any advance technique to exclude those key = value line? anyway, your expression is sufficient.
    – pinky
    Commented Jan 13, 2015 at 10:06
0

Your file is a csv file, so you could make your life easier and use the csv module:

import csv

f = open('data.txt', 'rb')

for row in csv.reader(f, delimiter=','):
    if row:
        item, key_and_val = row[3].split(':')
        item = item[2:]
        key, val = key_and_val.split('=')

        print item
        print '    {} => {}'.format(key.strip(), val.strip())

        for key_and_val in row[4:]:
            key, val = key_and_val.split('=')
            print '    {} => {}'.format(key.strip(), val.strip())

--output:--
RATIO
    casher# => 30
    Value => 2.579
    Units => ratio
    Error => N
RATIO
    casher# => 31
    Value => 4.509
    Units => ratio
    Error => N
RATIO
    casher# => 32
    Value => 3.735
    Units => ratio
    Error => N
RATIO
    casher# => 33
    Value => 2.401
    Units => ratio
    Error => N
CUSTOMER
    casher# => 30
    Value => 50
    Units => count
CUSTOMER
    casher# => 31
    Value => 6
    Units => count
CUSTOMER
    casher# => 32
    Value => 88
    Units => count
CUSTOMER
    casher# => 33
    Value => 33
    Units => count

your matching pattern also matched key=value even if the "item :" not exist, any advance technique to exclude those key = value line?

The following will skip lines without an item:

for row in csv.reader(f, delimiter=','):
    if row:
        if row[3].startswith('>>'):  #Check if there is an item
            item, key_and_val = row[3].split(': ')
            item = item[2:]
            key, val = key_and_val.split('=')
            print item
            print '    {} => {}'.format(key.strip(), val.strip())

            for key_and_val in row[4:]:
                key, val = key_and_val.split('=')
                print '    {} => {}'.format(key.strip(), val.strip())

f.close()
1
  • thank you @7stud, i will use your code after extracted them to csv file. ^^"
    – pinky
    Commented Jan 13, 2015 at 13:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.