892

I was wondering what the simplest way is to convert a string representation of a list like the following to a list:

x = '[ "A","B","C" , " D"]'

Even in cases where the user puts spaces in between the commas, and spaces inside of the quotes, I need to handle that as well and convert it to:

x = ["A", "B", "C", "D"] 

I know I can strip spaces with strip() and split() and check for non-letter characters. But the code was getting very kludgy. Is there a quick function that I'm not aware of?

0

21 Answers 21

1197
>>> import ast
>>> x = '[ "A","B","C" , " D"]'
>>> x = ast.literal_eval(x)
>>> x
['A', 'B', 'C', ' D']
>>> x = [n.strip() for n in x]
>>> x
['A', 'B', 'C', 'D']

ast.literal_eval:

Evaluate an expression node or a string containing only a Python literal or container display. The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, None and Ellipsis.

This can be used for evaluating strings containing Python values without the need to parse the values oneself. It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing.

6
  • 12
    Per comment below, this is dangerous as it simply runs whatever python is in the string. So if someone puts a call to delete everything in there, it happily will. Nov 18, 2017 at 21:15
  • 29
    @PaulKenjora: You're thinking of eval, not ast.literal_eval. Mar 19, 2018 at 23:29
  • 39
    ast.literal_eval is safer than eval, but it's not actually safe. As recent versions of the docs explain: "Warning It is possible to crash the Python interpreter with a sufficiently large/complex string due to stack depth limitations in Python’s AST compiler." It may, in fact, be possible to run arbitrary code via a careful stack-smashing attack, although as far as I know nobody's build a public proof of concept for that.
    – abarnert
    Mar 30, 2018 at 0:12
  • 1
    Well but what to do if the List does not have quotes? e.g. [4 of B, 1 of G]
    – sqp_125
    Apr 22, 2020 at 19:26
  • 4
    @sqp_125, then it's a regular list, and you don't need to parse anything?
    – ForceBru
    Jun 15, 2020 at 14:01
254

The json module is a better solution whenever there is a stringified list of dictionaries. The json.loads(your_data) function can be used to convert it to a list.

>>> import json
>>> x = '[ "A","B","C" , " D"]'
>>> json.loads(x)
['A', 'B', 'C', ' D']

Similarly

>>> x = '[ "A","B","C" , {"D":"E"}]'
>>> json.loads(x)
['A', 'B', 'C', {'D': 'E'}]
5
  • 32
    This works for ints but not for strings in my case because each string is single quoted not double quoted, sigh. Nov 18, 2017 at 21:16
  • 31
    As per @PaulKenjora's comment, it works for '["a","b"]' but not for "['a','b']". Jun 19, 2019 at 9:33
  • 5
    In my case I had to replace single quotes with double quotes in initial string to ensure it works .replace('\'', '"') But I was sure that data inside that string didn't contain any crucial single/double quotes in it that would affect the final result. Dec 10, 2020 at 9:23
  • If user should only enter list of numeric, I think this is the safest way to go to stop malicious intend user. Dec 12, 2021 at 5:27
  • The ast.literal_eval approach is more general. For example, JSON cannot handle b prefixes for strings, as it does not recognize a separate bytes type. JSON also requires double quotes for the strings. Aug 11, 2022 at 1:28
117

The eval is dangerous - you shouldn't execute user input.

If you have 2.6 or newer, use ast instead of eval:

>>> import ast
>>> ast.literal_eval('["A","B" ,"C" ," D"]')
["A", "B", "C", " D"]

Once you have that, strip the strings.

If you're on an older version of Python, you can get very close to what you want with a simple regular expression:

>>> x='[  "A",  " B", "C","D "]'
>>> re.findall(r'"\s*([^"]*?)\s*"', x)
['A', 'B', 'C', 'D']

This isn't as good as the ast solution, for example it doesn't correctly handle escaped quotes in strings. But it's simple, doesn't involve a dangerous eval, and might be good enough for your purpose if you're on an older Python without ast.

2
  • Could you please tell me what why did you say “The eval is dangerous - you shouldn’t execute user input.”? I am using 3.6 Jul 17, 2017 at 1:56
  • 2
    @AaryanDewan if you use eval directly, it will evaluate any valid python expression, which is potentially dangerous. literal_eval solves this problem by only evaluating Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None. Sep 21, 2017 at 23:28
30

Inspired from some of the answers above that work with base Python packages I compared the performance of a few (using Python 3.7.3):

Method 1: ast

import ast

list(map(str.strip, ast.literal_eval(u'[ "A","B","C" , " D"]')))
# ['A', 'B', 'C', 'D']

import timeit
timeit.timeit(stmt="list(map(str.strip, ast.literal_eval(u'[ \"A\",\"B\",\"C\" , \" D\"]')))", setup='import ast', number=100000)
# 1.292875313000195

Method 2: json

import json
list(map(str.strip, json.loads(u'[ "A","B","C" , " D"]')))
# ['A', 'B', 'C', 'D']

import timeit
timeit.timeit(stmt="list(map(str.strip, json.loads(u'[ \"A\",\"B\",\"C\" , \" D\"]')))", setup='import json', number=100000)
# 0.27833264000014424

Method 3: no import

list(map(str.strip, u'[ "A","B","C" , " D"]'.strip('][').replace('"', '').split(',')))
# ['A', 'B', 'C', 'D']

import timeit
timeit.timeit(stmt="list(map(str.strip, u'[ \"A\",\"B\",\"C\" , \" D\"]'.strip('][').replace('\"', '').split(',')))", number=100000)
# 0.12935059100027502

I was disappointed to see what I considered the method with the worst readability was the method with the best performance... there are trade-offs to consider when going with the most readable option... for the type of workloads I use Python for I usually value readability over a slightly more performant option, but as usual it depends.

2
  • 1
    is there any particular reason for there being a u in front of '[ "A","B","C" , " D"]' Nov 20, 2020 at 15:14
  • The manual method is simply not as powerful, and does less work, so it's not surprising that it's faster. It will not handle escape sequences in the strings, or a different quote type. (The JSON method demands double-quotes, but does process escape sequences.) It also will only process a flat list of strings; the other approaches can handle complex nested data structures. Aug 11, 2022 at 1:21
28

There is a quick solution:

x = eval('[ "A","B","C" , " D"]')

Unwanted whitespaces in the list elements may be removed in this way:

x = [x.strip() for x in eval('[ "A","B","C" , " D"]')]
3
  • 1
    this would still preserve the spaces inside the quotes
    – tosh
    Dec 12, 2009 at 18:26
  • 35
    This is an open invitation to arbitrary code execution, NEVER do this or anything like it unless you know with absolute certainty that the input will always be 100% trusted. Dec 12, 2009 at 18:29
  • 1
    I could use this suggestion because I knew my data was always gonna be in that format and was a data processing work. Mar 11, 2016 at 20:44
20
import ast
l = ast.literal_eval('[ "A","B","C" , " D"]')
l = [i.strip() for i in l]
16

If it's only a one dimensional list, this can be done without importing anything:

>>> x = u'[ "A","B","C" , " D"]'
>>> ls = x.strip('[]').replace('"', '').replace(' ', '').split(',')
>>> ls
['A', 'B', 'C', 'D']
2
  • 16
    Cautionary note: this could potentially be dangerous if any of the strings inside list has a comma in between. Oct 3, 2018 at 18:12
  • This will not work if your string list is a list of lists
    – crypdick
    Mar 19, 2020 at 15:37
12

There isn't any need to import anything or to evaluate. You can do this in one line for most basic use cases, including the one given in the original question.

One liner

l_x = [i.strip() for i in x[1:-1].replace('"',"").split(',')]

Explanation

x = '[ "A","B","C" , " D"]'
# String indexing to eliminate the brackets.
# Replace, as split will otherwise retain the quotes in the returned list
# Split to convert to a list
l_x = x[1:-1].replace('"',"").split(',')

Outputs:

for i in range(0, len(l_x)):
    print(l_x[i])
# vvvv output vvvvv
'''
 A
B
C
  D
'''
print(type(l_x)) # out: class 'list'
print(len(l_x)) # out: 4

You can parse and clean up this list as needed using list comprehension.

l_x = [i.strip() for i in l_x] # list comprehension to clean up
for i in range(0, len(l_x)):
    print(l_x[i])
# vvvvv output vvvvv
'''
A
B
C
D
'''

Nested lists

If you have nested lists, it does get a bit more annoying. Without using regex (which would simplify the replace), and assuming you want to return a flattened list (and the zen of python says flat is better than nested):

x = '[ "A","B","C" , " D", ["E","F","G"]]'
l_x = x[1:-1].split(',')
l_x = [i
    .replace(']', '')
    .replace('[', '')
    .replace('"', '')
    .strip() for i in l_x
]
# returns ['A', 'B', 'C', 'D', 'E', 'F', 'G']

If you need to retain the nested list it gets a bit uglier, but it can still be done just with regular expressions and list comprehension:

import re

x = '[ "A","B","C" , " D", "["E","F","G"]","Z", "Y", "["H","I","J"]", "K", "L"]'
# Clean it up so the regular expression is simpler
x = x.replace('"', '').replace(' ', '')
# Look ahead for the bracketed text that signifies nested list
l_x = re.split(r',(?=\[[A-Za-z0-9\',]+\])|(?<=\]),', x[1:-1])
print(l_x)
# Flatten and split the non nested list items
l_x0 = [item for items in l_x for item in items.split(',') if not '[' in items]
# Convert the nested lists to lists
l_x1 = [
    i[1:-1].split(',') for i in l_x if '[' in i
]
# Add the two lists
l_x = l_x0 + l_x1

This last solution will work on any list stored as a string, nested or not.

2
  • Notice the method doesn't play well with empty lists. You take '[]' and get back ['']. This might be an issue if you're parsing a column in a data frame. Nice solution otherwise! Jun 4, 2022 at 4:14
  • the list comprehension seems to bee slower than the x.strip('[]').replace('"', '').split(',') solution. Probably because the strip operation is repeated len(x) times instead of 1 and two list are created instead of 1 (the one returned by the 'split()`and the one returned by the comprehension).
    – Banane
    Sep 6, 2022 at 8:38
10

You can do this

**

x = '[ "A","B","C" , " D"]'
print(eval(x))

** best one is the accepted answer

Though this is not a safe way, the best answer is the accepted one. wasn't aware of the eval danger when answer was posted.

1
  • 3
    eval is not recommended in several places on this thread as it will simple run as code whatever is entered, presenting a security risk. it is also a duplicate answer.
    – born_naked
    Feb 4, 2022 at 16:53
6

Assuming that all your inputs are lists and that the double quotes in the input actually don't matter, this can be done with a simple regexp replace. It is a bit perl-y, but it works like a charm. Note also that the output is now a list of Unicode strings, you didn't specify that you needed that, but it seems to make sense given Unicode input.

import re
x = u'[ "A","B","C" , " D"]'
junkers = re.compile('[[" \]]')
result = junkers.sub('', x).split(',')
print result
--->  [u'A', u'B', u'C', u'D']

The junkers variable contains a compiled regexp (for speed) of all characters we don't want, using ] as a character required some backslash trickery. The re.sub replaces all these characters with nothing, and we split the resulting string at the commas.

Note that this also removes spaces from inside entries u'["oh no"]' ---> [u'ohno']. If this is not what you wanted, the regexp needs to be souped up a bit.

5

If you know that your lists only contain quoted strings, this pyparsing example will give you your list of stripped strings (even preserving the original Unicode-ness).

>>> from pyparsing import *
>>> x =u'[ "A","B","C" , " D"]'
>>> LBR,RBR = map(Suppress,"[]")
>>> qs = quotedString.setParseAction(removeQuotes, lambda t: t[0].strip())
>>> qsList = LBR + delimitedList(qs) + RBR
>>> print qsList.parseString(x).asList()
[u'A', u'B', u'C', u'D']

If your lists can have more datatypes, or even contain lists within lists, then you will need a more complete grammar - like this one in the pyparsing examples directory, which will handle tuples, lists, ints, floats, and quoted strings.

0
3

You may run into such problem while dealing with scraped data stored as Pandas DataFrame.

This solution works like charm if the list of values is present as text.

def textToList(hashtags):
    return hashtags.strip('[]').replace('\'', '').replace(' ', '').split(',')

hashtags = "[ 'A','B','C' , ' D']"
hashtags = textToList(hashtags)

Output: ['A', 'B', 'C', 'D']

No external library required.

2

This usually happens when you load list stored as string to CSV

If you have your list stored in CSV in form like OP asked:

x = '[ "A","B","C" , " D"]'

Here is how you can load it back to list:

import csv
with open('YourCSVFile.csv') as csv_file:
    reader = csv.reader(csv_file, delimiter=',')
    rows = list(reader)

listItems = rows[0]

listItems is now list

7
  • 1
    Not sure how this is related to the question... list(reader) gives a list of lists. Each inner list is a list of strings of the csv columns. There is no string representation of a list there to begin with...
    – Tomerikoo
    Apr 2, 2021 at 13:43
  • @Tomerikoo string representation of list is exactly the same only it's in the file.
    – Hrvoje
    Apr 2, 2021 at 14:27
  • No. A string representation of a list is "['1', '2', '3']". When you read a csv file with csv.reader, each line is ['1', '2', '3']. That is a list of strings. Not a string representation of a list...
    – Tomerikoo
    Apr 2, 2021 at 14:29
  • @Tomerikoo how about you store list in file and than use any method here to restore it.
    – Hrvoje
    Apr 2, 2021 at 14:40
  • 1
    Ok, let's say the csv has literally [1, 2, 3] inside it. Let's say a csv row is [1,2,3] 4 5. Reading it with list(reader) will give [["[1,2,3]", "4", "5"], ...] then doing rows[0] will give ["[1,2,3]", "4", "5"]. Again, I don't see how that answers the question...
    – Tomerikoo
    Apr 2, 2021 at 14:43
2

To further complete Ryan's answer using JSON, one very convenient function to convert Unicode is in this answer.

Example with double or single quotes:

>print byteify(json.loads(u'[ "A","B","C" , " D"]')
>print byteify(json.loads(u"[ 'A','B','C' , ' D']".replace('\'','"')))
['A', 'B', 'C', ' D']
['A', 'B', 'C', ' D']
1
  • The only new information here is a further processing step that is unrelated to the question that was asked, and also somewhere between irrelevant and harmful in most cases. The data generally should be understood as strings (unicode objects in 2.x), not byte sequences. Aug 11, 2022 at 1:27
1

json.loads() and json.dumps() from json package is the equivalent way of javascript JSON.parse() and JSON.stringify() so use json solution to keep life simpler

import json
a = '[ "A","B","C" , " D"]'
print(json.loads(a)) #['A', 'B', 'C', ' D']
b = ['A', 'B', 'C', ' D']
print(json.dumps(b)) # '["A", "B", "C", " D"]'

0

I would like to provide a more intuitive patterning solution with regex. The below function takes as input a stringified list containing arbitrary strings.

Stepwise explanation: You remove all whitespacing,bracketing and value_separators (provided they are not part of the values you want to extract, else make the regex more complex). Then you split the cleaned string on single or double quotes and take the non-empty values (or odd indexed values, whatever the preference).

def parse_strlist(sl):
import re
clean = re.sub("[\[\],\s]","",sl)
splitted = re.split("[\'\"]",clean)
values_only = [s for s in splitted if s != '']
return values_only

testsample: "['21',"foo" '6', '0', " A"]"

0

So, following all the answers I decided to time the most common methods:

from time import time
import re
import json

my_str = str(list(range(19)))
print(my_str)

reps = 100000

start = time()
for i in range(0, reps):
    re.findall("\w+", my_str)
print("Regex method:\t", (time() - start) / reps)

start = time()
for i in range(0, reps):
    json.loads(my_str)
print("JSON method:\t", (time() - start) / reps)

start = time()
for i in range(0, reps):
    ast.literal_eval(my_str)
print("AST method:\t\t", (time() - start) / reps)

start = time()
for i in range(0, reps):
    [n.strip() for n in my_str]
print("strip method:\t", (time() - start) / reps)

    regex method:     6.391477584838867e-07
    json method:     2.535374164581299e-06
    ast method:         2.4425282478332518e-05
    strip method:     4.983267784118653e-06

So in the end regex wins!

0

You can save yourself the .strip() function by just slicing off the first and last characters from the string representation of the list (see the third line below):

>>> mylist=[1,2,3,4,5,'baloney','alfalfa']
>>> strlist=str(mylist)
['1', ' 2', ' 3', ' 4', ' 5', " 'baloney'", " 'alfalfa'"]
>>> mylistfromstring=(strlist[1:-1].split(', '))
>>> mylistfromstring[3]
'4'
>>> for entry in mylistfromstring:
...     print(entry)
...     type(entry)
...
1
<class 'str'>
2
<class 'str'>
3
<class 'str'>
4
<class 'str'>
5
<class 'str'>
'baloney'
<class 'str'>
'alfalfa'
<class 'str'>
0

And with pure Python - not importing any libraries:

[x for x in  x.split('[')[1].split(']')[0].split('"')[1:-1] if x not in[',',' , ',', ']]
0

This is another solution if you don't want to import any library:

x = '[ "A","B","C" , " D"]'
def toList(stringList):
  stringList = stringList.split('[')[1]# removes "["
  stringList = stringList.split(']')[0]# removes "]"
  stringList = stringList.split(',')#gets objects in the list
  return [text.strip()[1:-1] for text in stringList] #eliminate additional " or ' in the string.
toList(x)

Output:

['A', 'B', 'C', ' D']

The caveat to this method is that it doesn't work if you have comma inside your string for example if your input is

x = '[ "A","B,F","C" , " D"]'

your output will be

['A', '', '', 'C', ' D']

which is not what you want.

-1

This solution is simpler than some I read in the previous answers, but it requires to match all features of the list.

x = '[ "A","B","C" , " D"]'
[i.strip() for i in x.split('"') if len(i.strip().strip(',').strip(']').strip('['))>0]

Output:

['A', 'B', 'C', 'D']

Not the answer you're looking for? Browse other questions tagged or ask your own question.