3

Suppose I have a string like this:

"key1=value1;key2=value2;key3=(key3.1=value3.1;key3.2=value3.2)"

I would like to get a dictionary corresponding to the above, where the value for key3 is the string

"(key3.1=value3.1;key3.2=value3.2)"

and eventually the corresponding sub-dictionary.

I know how to split the string at the semicolons, but how can I tell the parser to ignore the semicolon between parentheses?
This includes potentially nested parentheses.

Currently I am using an ad-hoc routine that looks for pairs of matching parentheses, "clears" its content, gets split positions and applies them to the original string, but this does not appear very elegant, there must be some prepackaged pythonic way to do this.

If anyone is interested, here is the code I am currently using:

def pparams(parameters, sep=';', defs='=', brc='()'):
    '''
    unpackages parameter string to struct
    for example, pippo(a=21;b=35;c=pluto(h=zzz;y=mmm);d=2d3f) becomes:
     a: '21'
     b: '35'
     c.fn: 'pluto'
     c.h='zzz'
     d: '2d3f'
     fn_: 'pippo'
    '''

    ob=strfind(parameters,brc[0])
    dp=strfind(parameters,defs)
    out={}

    if len(ob)>0:
        if ob[0]<dp[0]:
            #opening function
            out['fn_']=parameters[:ob[0]]
            parameters=parameters[(ob[0]+1):-1]
    if len(dp)>0:
        temp=smart_tokenize(parameters,sep,brc);
        for v in temp:
            defp=strfind(v,defs)
            pname=v[:defp[0]]
            pval=v[1+defp[0]:]
            if len(strfind(pval,brc[0]))>0:
                out[pname]=pparams(pval,sep,defs,brc);
            else:
                out[pname]=pval
    else:
        out['fn_']=parameters
    return out

def smart_tokenize( instr, sep=';', brc='()' ):
    '''
    tokenize string ignoring separators contained within brc
    '''
    tstr=instr;
    ob=strfind(instr,brc[0])
    while len(ob)>0:
        cb=findclsbrc(tstr,ob[0])
        tstr=tstr[:ob[0]]+'?'*(cb-ob[0]+1)+tstr[cb+1:]
        ob=strfind(tstr,brc[1])
    sepp=[-1]+strfind(tstr,sep)+[len(instr)+1]
    out=[]
    for i in range(1,len(sepp)):
        out.append(instr[(sepp[i-1]+1):(sepp[i])])
    return out

def findclsbrc(instr, brc_pos, brc='()'):
    '''
    given a string containing an opening bracket, finds the
    corresponding closing bracket
    '''
    tstr=instr[brc_pos:]
    o=strfind(tstr,brc[0])
    c=strfind(tstr,brc[1])
    p=o+c
    p.sort()
    s1=[1 if v in o else 0 for v in p]
    s2=[-1 if v in c else 0 for v in p]
    s=[s1v+s2v for s1v,s2v in zip(s1,s2)]
    s=[sum(s[:i+1]) for i in range(len(s))] #cumsum
    return p[s.index(0)]+brc_pos

def strfind(instr, substr):
    '''
    returns starting position of each occurrence of substr within instr
    '''
    i=0
    out=[]
    while i<=len(instr):
        try:
            p=instr[i:].index(substr)
            out.append(i+p)
            i+=p+1
        except:
            i=len(instr)+1
    return out
1
  • 6
    It would be much easier to use a standard persistence format Commented Feb 14, 2012 at 14:23

3 Answers 3

6

If you want to build a real parser, use one of the Python parsing libraries, like PLY or PyParsing. If you figure such a full-fledged library is overkill for the task at hand, go for some hack like the one you already have. I'm pretty sure there is no clean few-line solution without an external library.

5
  • I'm pretty sure it's doable with some not too complex regex. Edit: Kasey Kirkham beat me to it!
    – Voo
    Commented Feb 14, 2012 at 15:49
  • @Voo: I never said it's impossible. Parsing nested parentheses with regular expressions is always a hack, though. Try to add quoting for keys or values, and things will become pretty bad. Commented Feb 14, 2012 at 15:54
  • You can do it though with some tricks (missed the part about nesting in the original post!) - thanks to the "not so much regular anymore regexes". Personally I'd go with a mixed regex and iterative solution, mostly because it seems a bit overkill to use a fullblown parsing library for a few lines of code.
    – Voo
    Commented Feb 14, 2012 at 16:04
  • @Voo: Personally, I'd just use JSON. Commented Feb 14, 2012 at 16:12
  • Clearly the best solution if we can change the format. Either that or if performance was important I'd go with protobuffs.
    – Voo
    Commented Feb 14, 2012 at 16:26
1

Expanding on Sven Marnach's answer, here's an example of a pyparsing grammar that should work for you:

from pyparsing import (ZeroOrMore, Word, printables, Forward,
                       Group, Suppress, Dict)


collection = Forward()
simple_value = Word(printables, excludeChars='()=;')
key = simple_value
inner_collection = Suppress('(') + collection + Suppress(')')
value = simple_value ^ inner_collection
key_and_value = Group(key + Suppress('=') + value)
collection << Dict(key_and_value + ZeroOrMore(Suppress(';') + key_and_value))


coll = collection.parseString(
    "key1=value1;key2=value2;key3=(key3.1=value3.1;key3.2=value3.2)")

print coll['key1']             # value1
print coll['key2']             # value2
print coll['key3']['key3.1']   # value3.1
0

You could use a regex to capture the groups:

>>> import re
>>> s = "key1=value1;key2=value2;key3=(key3.1=value3.1;key3.2=value3.2)"
>>> r = re.compile('(\w+)=(\w+|\([^)]+\));?')
>>> dict(r.findall(s))

This regex says:

(\w)+ # Find and capture a group with 1 or more word characters (letters, digits, underscores)
= # Followed by the literal character '='
(\w+ # Followed by a group with 1 or more word characters
|\([^)]+\) # or a group that starts with an open paren (parens escaped with '\(' or \')'), followed by anything up until a closed paren, which terminates the alternate grouping
);? # optionally this grouping might be followed by a semicolon.

Gotta say, kind of a strange grammar. You should consider using a more standard format. If you need guidance choosing one maybe ask another question. Good luck!

2
  • I think you should pass re.VERBOSE to your regex and then include all your comments in the actual regex. Makes them much more readable and is a feature that's really underused but imho essential when using them. Otherwise very nice!
    – Voo
    Commented Feb 14, 2012 at 15:51
  • 2
    This stops working as soon as you add another level of nesting. Parsing nested parentheses with regular expressions is only possible by starting from the inner-most expression and working one's way up the parse tree. It will always be hacky, so I don't see how this should be better than what the OP already has. Commented Feb 14, 2012 at 15:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.