1182

How can I convert the str representation of a dictionary, such as the following string, into a dict?

s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"

I prefer not to use eval. What else can I use?

2
  • 2
    If you can't use Python 2.6, you can use a simple safeeval implmenentation like code.activestate.com/recipes/364469 It piggybacks on the Python compiler so you don't have to do all the gross work yourself. Commented Jun 12, 2009 at 19:09
  • 30
    Note: For those that come here with deceptively similar looking JSON data, you want to go read Parse JSON in Python instead. JSON is not the same thing as Python. If you have " double quotes around your strings you probably have JSON data. You can also look for null, true or false, Python syntax uses None, True and False. Commented Nov 22, 2018 at 13:23

13 Answers 13

1714

You can use the built-in ast.literal_eval:

>>> import ast
>>> ast.literal_eval("{'muffin' : 'lolz', 'foo' : 'kitty'}")
{'muffin': 'lolz', 'foo': 'kitty'}

This is safer than using eval. As its own docs say:

>>> help(ast.literal_eval)
Help on function literal_eval in module ast:

literal_eval(node_or_string)
    Safely evaluate an expression node or a string containing a Python
    expression.  The string or node provided may only consist of the following
    Python literal structures: strings, numbers, tuples, lists, dicts, booleans,
    and None.

For example:

>>> eval("shutil.rmtree('mongo')")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1, in <module>
  File "/opt/Python-2.6.1/lib/python2.6/shutil.py", line 208, in rmtree
    onerror(os.listdir, path, sys.exc_info())
  File "/opt/Python-2.6.1/lib/python2.6/shutil.py", line 206, in rmtree
    names = os.listdir(path)
OSError: [Errno 2] No such file or directory: 'mongo'
>>> ast.literal_eval("shutil.rmtree('mongo')")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/Python-2.6.1/lib/python2.6/ast.py", line 68, in literal_eval
    return _convert(node_or_string)
  File "/opt/Python-2.6.1/lib/python2.6/ast.py", line 67, in _convert
    raise ValueError('malformed string')
ValueError: malformed string
10
  • 1
    I should add that you need to sanitize the string for use with ast.literal_eval. (ensure quotes/double quotes in string are escaped) Commented Oct 4, 2012 at 10:10
  • i get this error I am on python 2.6 (x86) on windows 7 x64 File "D:\Python26\lib\ast.py", line 48, in literal_eval node_or_string = parse(node_or_string, mode='eval') File "D:\Python26\lib\ast.py", line 36, in parse return compile(expr, filename, mode, PyCF_ONLY_AST) File "<unknown>", line 1 ^ SyntaxError: invalid syntax
    – user1176501
    Commented Dec 10, 2012 at 7:38
  • 2
    what about "dict(a=1)" style strings?
    – n611x007
    Commented Jul 4, 2014 at 11:44
  • 4
    why don't use json.dumps and json.loads insead, I found this solution more elevant thant using eval
    – JuanB
    Commented Jan 7, 2018 at 16:23
  • 6
    @JuanB json.loads doesn't accept single-quoted strings. It also doesn't accept None, only null. Commented Apr 22, 2021 at 14:55
419

https://docs.python.org/library/json.html

JSON can solve this problem, though its decoder wants double quotes around keys and values. If you don't mind a replace hack...

import json
s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"
json_acceptable_string = s.replace("'", "\"")
d = json.loads(json_acceptable_string)
# d = {u'muffin': u'lolz', u'foo': u'kitty'}

NOTE that if you have single quotes as a part of your keys or values this will fail due to improper character replacement. This solution is only recommended if you have a strong aversion to the eval solution.

More about json single quote: jQuery.parseJSON throws “Invalid JSON” error due to escaped single quote in JSON

9
  • 5
    Another problem is for "{0: 'Hello'}". Commented Aug 23, 2017 at 12:14
  • 5
    This also fails if you have trailing commas (not JSON compliant), eg: "{'muffin' : 'lolz', 'foo' : 'kitty',}" Commented Mar 21, 2018 at 16:35
  • 3
    Single-quoted strings, tuple literals, and trailing commas are not valid JSON. json.loads will only work on a valid JSON string. See the spec here: json.org Using json.loads is the safest solution, so use if possible. I would recommend transforming your input into valid JSON if necessary. Commented May 22, 2018 at 19:31
  • 1
    Also this solution does not work if you have unicode strings Commented Sep 20, 2018 at 13:06
  • 6
    Not working if value None (without single quotes, so can' t replace), e.g. "{'d': None}"
    – 林果皞
    Commented Aug 9, 2020 at 1:33
241

using json.loads:

>>> import json
>>> h = '{"foo":"bar", "foo2":"bar2"}'
>>> d = json.loads(h)
>>> d
{u'foo': u'bar', u'foo2': u'bar2'}
>>> type(d)
<type 'dict'>
8
  • 27
    I dont think it answers the OP's answer. How do we use json.laads to convert a string s = "{'muffin' : 'lolz', 'foo' : 'kitty'}" to dict?
    – technazi
    Commented May 13, 2016 at 14:28
  • 4
    @technazi: json.loads(h.replace("'",'"'))
    – ntg
    Commented Dec 8, 2016 at 15:08
  • 1
    However, there are limits, e.g.: h= '{"muffin" : "lolz", "foo" : "kitty",}', also h= '{"muffin's" : "lolz", "foo" : "kitty"}', (just noticed part of the same comments in a similar answer... still leaving here for completeness...)
    – ntg
    Commented Dec 8, 2016 at 15:29
  • 8
    In my opinion, that's the shortest and easiest way... Definitely the one I personally prefer. Commented Jan 11, 2017 at 8:40
  • 3
    @nostradamus Too many exceptions, floating point values, tuples, etc. etc.
    – MrR
    Commented Feb 23, 2019 at 0:04
55

To OP's example:

s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"

We can use Yaml to deal with this kind of non-standard json in string:

>>> import yaml
>>> s = "{'muffin' : 'lolz', 'foo' : 'kitty'}"
>>> s
"{'muffin' : 'lolz', 'foo' : 'kitty'}"
>>> yaml.load(s)
{'muffin': 'lolz', 'foo': 'kitty'}
4
  • 9
    This will cause 'yes' and 'no' strings to be converted to True / False Commented Aug 18, 2017 at 17:57
  • i got my value that works fine....but i get a error with it "AMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read msg.pyyaml.org/load for full details." what is it??
    – shailu
    Commented May 2, 2021 at 17:13
  • 1
    Only use this yaml parser for trusted input. Preferably use safe_load to avoid security implications. Commented Jul 23, 2021 at 13:44
  • config = yaml.load(ymlfile, Loader=yaml.Loader) Commented Nov 25, 2022 at 6:18
36

To summarize:

import ast, yaml, json, timeit

descs=['short string','long string']
strings=['{"809001":2,"848545":2,"565828":1}','{"2979":1,"30581":1,"7296":1,"127256":1,"18803":2,"41619":1,"41312":1,"16837":1,"7253":1,"70075":1,"3453":1,"4126":1,"23599":1,"11465":3,"19172":1,"4019":1,"4775":1,"64225":1,"3235":2,"15593":1,"7528":1,"176840":1,"40022":1,"152854":1,"9878":1,"16156":1,"6512":1,"4138":1,"11090":1,"12259":1,"4934":1,"65581":1,"9747":2,"18290":1,"107981":1,"459762":1,"23177":1,"23246":1,"3591":1,"3671":1,"5767":1,"3930":1,"89507":2,"19293":1,"92797":1,"32444":2,"70089":1,"46549":1,"30988":1,"4613":1,"14042":1,"26298":1,"222972":1,"2982":1,"3932":1,"11134":1,"3084":1,"6516":1,"486617":1,"14475":2,"2127":1,"51359":1,"2662":1,"4121":1,"53848":2,"552967":1,"204081":1,"5675":2,"32433":1,"92448":1}']
funcs=[json.loads,eval,ast.literal_eval,yaml.load]

for  desc,string in zip(descs,strings):
    print('***',desc,'***')
    print('')
    for  func in funcs:
        print(func.__module__+' '+func.__name__+':')
        %timeit func(string)        
    print('')

Results:

*** short string ***

json loads:
4.47 µs ± 33.4 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
builtins eval:
24.1 µs ± 163 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
ast literal_eval:
30.4 µs ± 299 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
yaml load:
504 µs ± 1.29 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

*** long string ***

json loads:
29.6 µs ± 230 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
builtins eval:
219 µs ± 3.92 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
ast literal_eval:
331 µs ± 1.89 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
yaml load:
9.02 ms ± 92.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Conclusion: prefer json.loads

2
  • 14
    Except this won't work with his single-quoted string, which was part of his initial problem. Performance was never mentioned. Commented Nov 26, 2018 at 16:49
  • 5
    +1 for the benchmarks (it helps making an informed decision), -1 for the conclusion: as mentioned many times, json fails in many cases. Should be up to the user to choose between features vs performance.
    – MestreLion
    Commented Apr 19, 2021 at 12:27
25

If the string can always be trusted, you could use eval (or use literal_eval as suggested; it's safe no matter what the string is.) Otherwise you need a parser. A JSON parser (such as simplejson) would work if he only ever stores content that fits with the JSON scheme.

2
  • 9
    Starting in 2.6, simplejson is included in the Python standard library as the json module. Commented Jun 12, 2009 at 18:37
  • 12
    Yeah, that's a good answer, but note that officially JSON doesn't support single-quoted strings, as given in the original poster's example.
    – Ben Hoyt
    Commented Jun 14, 2009 at 21:30
20

Use json. the ast library consumes a lot of memory and and slower. I have a process that needs to read a text file of 156Mb. Ast with 5 minutes delay for the conversion dictionary json and 1 minutes using 60% less memory!

1
  • 15
    but has its limits: try converting the string "{'foo':'bar',}"
    – ntg
    Commented Dec 8, 2016 at 15:04
11
string = "{'server1':'value','server2':'value'}"

#Now removing { and }
s = string.replace("{" ,"")
finalstring = s.replace("}" , "")

#Splitting the string based on , we get key value pairs
list = finalstring.split(",")

dictionary ={}
for i in list:
    #Get Key Value pairs separately to store in dictionary
    keyvalue = i.split(":")

    #Replacing the single quotes in the leading.
    m= keyvalue[0].strip('\'')
    m = m.replace("\"", "")
    dictionary[m] = keyvalue[1].strip('"\'')

print dictionary
1
  • 3
    Many mistakes in this approach. What if value of a key contains { or }. What if it is nested dict. What if value contains , ??
    – Om Sao
    Commented Mar 29, 2019 at 10:28
7

Optimized code of Siva Kameswara Rao Munipalle

s = s.replace("{", "").replace("}", "").split(",")
            
dictionary = {}

for i in s:
    dictionary[i.split(":")[0].strip('\'').replace("\"", "")] = i.split(":")[1].strip('"\'')
            
print(dictionary)
6

no any libs are used (python2):

dict_format_string = "{'1':'one', '2' : 'two'}"
d = {}
elems  = filter(str.isalnum,dict_format_string.split("'"))
values = elems[1::2]
keys   = elems[0::2]
d.update(zip(keys,values))

NOTE: As it has hardcoded split("'") will work only for strings where data is "single quoted".

NOTE2: In python3 you need to wrap filter() to list() to get list.

1
  • elems = filter(str.isalnum,dict_format_string.split("'")) should be list(elems = filter(str.isalnum,dict_format_string.split("'"))) without converting to list it would still be 'filter' object Commented Jan 23, 2021 at 22:08
2

I couldn't use any of the above answers cause I had a string with the dtypes specified, so I used the json.load as follow:

string_dict = """{'contexts': array(['Programmed cell death (PCD) is the regulated death of cells within an organism.', ..., '...'], dtype=object)},..."""

# Replace array( with np.array(
string_dict = string_dict.replace("array(", "np.array(")

# Evaluate the string as python code
python_dict = ast.literal_eval(string_dict)
2

If you have a string representation of a Python object that cannot be parsed by ast.literal_eval or json, then try asteval module. You can install it via pip first to use it: pip install asteval.

Its main advantage over the built-in ast or eval is that it can evaluate string representations of complex Python objects much easier than ast and much more safely than eval. Use it especially if your string contains math expressions such as nan, Decimal, numpy objects etc.

To use it, we must instantiate an Interpreter object and call it with the string to evaluate. In the example below, the string representation of the dictionary which is not JSON and contains NaN which cannot be converted by ast.literal_eval; however, asteval.Interpreter evaluates it correctly.

import ast
import json
from asteval import Interpreter

s = "{1: nan, 2: 3}"
ast.literal_eval(s)     # ValueError: malformed node or string
json.loads(s)           # JSONDecodeError
aeval = Interpreter()
aeval(s)                # {1: nan, 2: 3}

Some other examples where literal_eval or json.loads fails but asteval works.

  1. If you have the string representation of numpy objects and if numpy is installed on your system, then it's much easier to convert to the proper object with asteval as well.

    aeval = Interpreter()
    aeval("{'a': array([ 1.,  2., nan])}")    # {'a': array([ 1.,  2., nan])}
    
  2. By default, asteval evaluates numpy functions; however, if you want to make it parse some symbols in a certain way (e.g. numpy dtypes), you can define a custom symbol table and pass it to Interpreter at creation.

    Below is an example where a symbol table that defines how to evaluate numpy dtypes, decimal.Decimal type and null value is created and is passed to the Interpreter.

    from asteval import Interpreter, make_symbol_table
    from decimal import Decimal
    
    s = "{'a': array(['text'], dtype=object), 'b': Decimal('3.33'), 'c': null, 'd': array([1, 2], dtype=int8)}"
    symtable = make_symbol_table(object=object, Decimal=Decimal, null=None, int8='int8')
    aeval = Interpreter(symtable=symtable)
    d = aeval(s)
    print(d)   # {'a': array(['text'], dtype=object), 'b': Decimal('3.33'), 'c': None, 'd': array([1, 2], dtype=int8)}
    

Finally, keep in mind that since it evaluates any numpy/math functions (really any function that is fed into it as long as it is defined in the symtable argument), please be mindful of the input you pass into it.

0

My string didn't have quotes inside:
s = 'Date: 2022-11-29T10:57:01.024Z, Size: 910.11 KB'

My solution was to use str.split:
{k:v for k, v in map(lambda d: d.split(': '), s.split(', '))}

2
  • @EricAya, why did you edit my question out? How can I do this without a map?
    – A. West
    Commented Nov 29, 2022 at 18:17
  • Because the answers section is just for answers, not for questions, this is not a discussion forum. If you have a new question you need to post an actual new question.
    – Eric Aya
    Commented Nov 29, 2022 at 22:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.