232

The typical ConfigParser generated file looks like:

[Section]
bar=foo
[Section 2]
bar2= baz

Now, is there a way to index lists like, for instance:

[Section 3]
barList={
    item1,
    item2
}

Related question: Python’s ConfigParser unique keys per section

1
  • 3
    Remove the curly braces and the comma and then use config['Section 3']['barList'].split().
    – aderchox
    Nov 19, 2021 at 12:59

19 Answers 19

278

I am using a combination of ConfigParser and JSON:

[Foo]
fibs: [1,1,2,3,5,8,13]

just read it with:

>>> json.loads(config.get("Foo","fibs"))
[1, 1, 2, 3, 5, 8, 13]

You can even break lines if your list is long (thanks @peter-smit):

[Bar]
files_to_check = [
     "/path/to/file1",
     "/path/to/file2",
     "/path/to/another file with space in the name"
     ]

Of course i could just use JSON, but i find config files much more readable, and the [DEFAULT] Section very handy.

10
  • 1
    It's awesome because it does automatically "cast" values which can be useful if you don't know the types beforehand.
    – LeGBT
    Oct 26, 2014 at 21:26
  • 6
    You will have to have ["a", "b", "c"] for strings for them to work. For me, this clicks for numbers but as cfg files are mostly editable - adding "" everytime is a pain. I'd rather use comma and then split it. Feb 16, 2015 at 15:36
  • 3
    how would this work for raw strings, e.g. key5 : [r"abc $x_i$", r"def $y_j$"] ? They raise the error json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)
    – kingusiu
    Apr 9, 2019 at 18:09
  • 1
    Python 3.6 raise error: json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)
    – Luke
    Aug 14, 2020 at 14:55
  • 1
    You must use " (double quotes) in the JSON string for string values. json.loads('{"key": "value"}') and not ' (single quotes) or you will get a JSONDecodeError.
    – Neal
    Dec 17, 2020 at 21:56
167

There is nothing stopping you from packing the list into a delimited string and then unpacking it once you get the string from the config. If you did it this way your config section would look like:

[Section 3]
barList=item1,item2

It's not pretty but it's functional for most simple lists.

6
  • 3
    And if you've got complex lists, you can refer to this question: stackoverflow.com/questions/330900/… :-)
    – John Fouhy
    Dec 3, 2008 at 1:06
  • 1
    @wim See my answer, you can use \n as delimiter
    – Peter Smit
    Aug 8, 2012 at 14:27
  • @wim You would need to implement a way to escape the delimiter character if it can be a legal character. (And a way to escape whatever character you use for escaping.)
    – jamesdlin
    Aug 29, 2017 at 3:25
  • 2
    What if a list has a single element? Apr 23, 2019 at 19:50
  • 1
    Sounds like a great pull request
    – malanb5
    Mar 27, 2020 at 17:44
112

I recently implemented this with a dedicated section in a config file for a list:

[paths]
path1           = /some/path/
path2           = /another/path/
...

and using config.items( "paths" ) to get an iterable list of path items, like so:

path_items = config.items( "paths" )
for key, path in path_items:
    #do something with path
5
  • 4
    I like this solution, because you can ; comment out certain items from the list without having to rewrite the whole list.
    – wim
    Feb 27, 2012 at 23:24
  • 1
    +1, but if you do this, just be careful with also using key, as ConfigParser converts all such keys to lower-case
    – Alex Dean
    May 1, 2012 at 18:49
  • 4
    @AlexDean You can setup the ConfigParser to leave the camelCase in place by setting optionxform = str. Example: config = ConfigParser.SafeConfigParser() config.optionxform = str Then the case will be left alone Sep 13, 2012 at 21:06
  • 1
    @Henry Cooke Have you tested that when a key is listed multiple times?
    – DevPlayer
    Oct 22, 2016 at 2:08
  • 3
    @DevPlayer With multi-key usage you only get the last value. (responding to 2 yr old comment for benefit of other readers)
    – Marcin K
    Nov 18, 2018 at 17:45
81

One thing a lot of people don't know is that multi-line configuration-values are allowed. For example:

;test.ini
[hello]
barlist = 
    item1
    item2

The value of config.get('hello','barlist') will now be:

"\nitem1\nitem2"

Which you easily can split with the splitlines method (don't forget to filter empty items).

If we look to a big framework like Pyramid they are using this technique:

def aslist_cronly(value):
    if isinstance(value, string_types):
        value = filter(None, [x.strip() for x in value.splitlines()])
    return list(value)

def aslist(value, flatten=True):
    """ Return a list of strings, separating the input based on newlines
    and, if flatten=True (the default), also split on spaces within
    each line."""
    values = aslist_cronly(value)
    if not flatten:
        return values
    result = []
    for value in values:
        subvalues = value.split()
        result.extend(subvalues)
    return result

Source

Myself, I would maybe extend the ConfigParser if this is a common thing for you:

class MyConfigParser(ConfigParser):
    def getlist(self,section,option):
        value = self.get(section,option)
        return list(filter(None, (x.strip() for x in value.splitlines())))

    def getlistint(self,section,option):
        return [int(x) for x in self.getlist(section,option)]

Note that there are a few things to look out for when using this technique

  1. New lines that are items should start with whitespace (e.g. a space or a tab)
  2. All following lines that start with whitespace are considered to be part of the previous item. Also if it has an = sign or if it starts with a ; following the whitespace.
3
  • Why do you use .splitlines() instead of .split()? Using default behavior of each, split is clearly superior (filters out blank lines). Unless I'm missing something...
    – rsaw
    Jan 13, 2015 at 20:13
  • 9
    .split() breaks on all whitespace (unless a specific character is given), .splitlines() breaks on all newline characters.
    – Peter Smit
    Jan 14, 2015 at 4:56
  • Ahhh good point. I didn't think about that as none of my values had spaces.
    – rsaw
    Jan 14, 2015 at 14:30
60

No mention of the converters kwarg for ConfigParser() in any of these answers was rather disappointing.

According to the documentation you can pass a dictionary to ConfigParser that will add a get method for both the parser and section proxies. So for a list:

example.ini

[Germ]
germs: a,list,of,names, and,1,2, 3,numbers

Parser example:

cp = ConfigParser(converters={'list': lambda x: [i.strip() for i in x.split(',')]})
cp.read('example.ini')
cp.getlist('Germ', 'germs')
['a', 'list', 'of', 'names', 'and', '1', '2', '3', 'numbers']
cp['Germ'].getlist('germs')
['a', 'list', 'of', 'names', 'and', '1', '2', '3', 'numbers']

This is my personal favorite as no subclassing is necessary and I don't have to rely on an end user to perfectly write JSON or a list that can be interpreted by ast.literal_eval.

1
  • 3
    Best answer here. It uses what ConfigParser already gives. No additional package, method, class… Jan 28, 2021 at 15:12
57

If you want to literally pass in a list then you can use:

ast.literal_eval()

For example configuration:

[section]
option=["item1","item2","item3"]

The code is:

import ConfigParser
import ast

my_list = ast.literal_eval(config.get("section", "option"))
print(type(my_list))
print(my_list)

output:

<type'list'>
["item1","item2","item3"]
3
  • 3
    In this case, what is the advantage of using ast.literal_eval() when comparing to use the (arguably more popular) json.loads()? I think the latter provides more security, no?
    – RayLuo
    Jan 25, 2016 at 22:34
  • 3
    I would love to see and example of this, feel free to add an answer to this thread if you feel it would help, although your comment would make a good question in itself. The answer I gave simplifies the consumption of lists from ConfigParser so is internal to the app removing the comlication of using regex. I could not comment on its "secuity" value without context. Jan 27, 2016 at 8:47
  • I would be careful using literal_eval which expect python string after = or : hence you cannot use anymore e.g. path1 = /some/path/ but path1 = '/some/path/'
    – vldbnc
    Feb 19, 2019 at 14:22
15

I landed here seeking to consume this...

[global]
spys = richard.sorge@cccp.gov, mata.hari@deutschland.gov

The answer is to split it on the comma and strip the spaces:

SPYS = [e.strip() for e in parser.get('global', 'spys').split(',')]

To get a list result:

['richard.sorge@cccp.gov', 'mata.hari@deutschland.gov']

It may not answer the OP's question exactly but might be the simple answer some people are looking for.

2
  • 3
    I thought Dick was at sorger@espionage.su! No wonder my mail kept bouncing! >_<
    – Augusta
    Apr 20, 2016 at 21:25
  • 1
    Reading this comment 4 years later and chuckling at the easter egg Jun 23, 2020 at 7:26
14

This is what I use for lists:

config file content:

[sect]
alist = a
        b
        c

code :

l = config.get('sect', 'alist').split('\n')

it work for strings

in case of numbers

config content:

nlist = 1
        2
        3

code:

nl = config.get('sect', 'alist').split('\n')
l = [int(nl) for x in nl]

thanks.

2
  • This is the one that I was actually looking for thanks @LittleEaster
    – ashley
    Sep 6, 2019 at 15:12
  • Shouldn't the last line be l = [int(x) for x in nl] ? Otherwise x isn't used and an error is returned: TypeError: int() argument must be a string, a bytes-like object or a number, not 'list' Apr 28, 2021 at 12:56
8

So another way, which I prefer, is to just split the values, for example:

#/path/to/config.cfg
[Numbers]
first_row = 1,2,4,8,12,24,36,48

Could be loaded like this into a list of strings or integers, as follows:

import configparser

config = configparser.ConfigParser()
config.read('/path/to/config.cfg')

# Load into a list of strings
first_row_strings = config.get('Numbers', 'first_row').split(',')

# Load into a list of integers
first_row_integers = [int(x) for x in config.get('Numbers', 'first_row').split(',')]

This method prevents you from needing to wrap your values in brackets to load as JSON.

3
  • Hi Mitch, in the latter case wouldn't have been nicer to use get_int('first_row').split(',') instead of explicitly convert it to int while looping?
    – Guido
    Dec 11, 2019 at 9:41
  • @Guido - Did you try your suggestion? Did you mean this: first_row_integers = [x for x in config.getint('Numbers', 'first_row').split(',')] ? It gives an error: ValueError: invalid literal for int() with base 10: '1,2,4,8,12,24,36,48' Apr 28, 2021 at 12:47
  • @Greenonline true, now I don't recall where I was using it but for sure I must have not used split at the end otherwise it meant I provided a string and I did not parse immediately to integers.
    – Guido
    Apr 29, 2021 at 13:07
8

I completed similar task in my project with section with keys without values:

import configparser

# allow_no_value param says that no value keys are ok
config = configparser.ConfigParser(allow_no_value=True)

# overwrite optionxform method for overriding default behaviour (I didn't want lowercased keys)
config.optionxform = lambda optionstr: optionstr

config.read('./app.config')

features = list(config['FEATURES'].keys())

print(features)

Output:

['BIOtag', 'TextPosition', 'IsNoun', 'IsNomn']

app.config:

[FEATURES]
BIOtag
TextPosition
IsNoun
IsNomn
4

Only primitive types are supported for serialization by config parser. I would use JSON or YAML for that kind of requirement.

2
  • thanks for the clarification, utku. the only problem is that i can't use external packages at the moment. i think i'm gonna write a simple class to handle this. i'll share it eventually.
    – pistacchio
    Dec 2, 2008 at 22:44
  • What version of Python are you running? The JSON module is included with 2.6. Dec 3, 2008 at 0:11
3

To take Grr's answer (my favorite) a step further, instead of enclosing list items in quotes in the .ini file, you can use the map function. This allows you to pythonically specify list item datatypes.

Config file:

[section]
listKey1: 1001, 1002, 1003
listKey2: AAAA, BBBB, CCCC

Code:

cfgFile = 'config.ini'
parser = ConfigParser(converters={'list': lambda x: [i.strip() for i in x.split(',')]})
parser.read(cfgFile)

list1 = list(map(int, parser.getlist('section', 'listKey1')))
list2 = list(map(str, parser.getlist('section', 'listKey2')))

print(list1)
print(list2)

Output:

[1001, 1002, 1003]
['AAAA', 'BBBB', 'CCCC']
2

If this is your config.ini:

[Section 3]
barList=item1,item2

Then with configparser you could do this:

from configparser import ConfigParser
config = ConfigParser()
config.read('config.ini')
my_list = config['Section 3']['barList'].split(',')

You will get:

 my_list =  ['item1', 'item2']

The split()-method will return a list, see Python string docs.

If you have white spaces in your config.ini like this:

[Section 3]
barList= item1, item2

Then you'd better do this:

my_list = [x.strip() for x in config['Section 3']['barList'].split(',')]

If your items are numbers (integers for instance), just apply:

my_list_of_ints = list(map(int, my_list))

You will get:

my_list_of_ints =  [item1, item2]
1

I faced the same problem in the past. If you need more complex lists, consider creating your own parser by inheriting from ConfigParser. Then you would overwrite the get method with that:

    def get(self, section, option):
    """ Get a parameter
    if the returning value is a list, convert string value to a python list"""
    value = SafeConfigParser.get(self, section, option)
    if (value[0] == "[") and (value[-1] == "]"):
        return eval(value)
    else:
        return value

With this solution you will also be able to define dictionaries in your config file.

But be careful! This is not as safe: this means anyone could run code through your config file. If security is not an issue in your project, I would consider using directly python classes as config files. The following is much more powerful and expendable than a ConfigParser file:

class Section
    bar = foo
class Section2
    bar2 = baz
class Section3
    barList=[ item1, item2 ]
1
  • I was thinking of doing this, however: why not have the config values set up like barList=item1,item2 and then call if value.find(',') > 0: return value.split(','), or better yet, have the application parse all config options as lists, and just .split(',') everything blindly?
    – yurisich
    Oct 31, 2012 at 16:53
1
import ConfigParser
import os

class Parser(object):
    """attributes may need additional manipulation"""
    def __init__(self, section):
        """section to retun all options on, formatted as an object
        transforms all comma-delimited options to lists
        comma-delimited lists with colons are transformed to dicts
        dicts will have values expressed as lists, no matter the length
        """
        c = ConfigParser.RawConfigParser()
        c.read(os.path.join(os.path.dirname(__file__), 'config.cfg'))

        self.section_name = section

        self.__dict__.update({k:v for k, v in c.items(section)})

        #transform all ',' into lists, all ':' into dicts
        for key, value in self.__dict__.items():
            if value.find(':') > 0:
                #dict
                vals = value.split(',')
                dicts = [{k:v} for k, v in [d.split(':') for d in vals]]
                merged = {}
                for d in dicts:
                    for k, v in d.items():
                        merged.setdefault(k, []).append(v)
                self.__dict__[key] = merged
            elif value.find(',') > 0:
                #list
                self.__dict__[key] = value.split(',')

So now my config.cfg file, which could look like this:

[server]
credentials=username:admin,password:$3<r3t
loggingdirs=/tmp/logs,~/logs,/var/lib/www/logs
timeoutwait=15

Can be parsed into fine-grained-enough objects for my small project.

>>> import config
>>> my_server = config.Parser('server')
>>> my_server.credentials
{'username': ['admin'], 'password', ['$3<r3t']}
>>> my_server.loggingdirs:
['/tmp/logs', '~/logs', '/var/lib/www/logs']
>>> my_server.timeoutwait
'15'

This is for very quick parsing of simple configs, you lose all ability to fetch ints, bools, and other types of output without either transforming the object returned from Parser, or re-doing the parsing job accomplished by the Parser class elsewhere.

0

json.loads & ast.literal_eval seems to be working but simple list within config is treating each character as byte so returning even square bracket....

meaning if config has fieldvalue = [1,2,3,4,5]

then config.read(*.cfg) config['fieldValue'][0] returning [ in place of 1

0

As mentioned by Peter Smit (https://stackoverflow.com/a/11866695/7424596) You might want to extend ConfigParser, in addition, an Interpolator can be used to automatically convert into and from the list.

For reference at the bottom you can find code which automatically converts config like:

[DEFAULT]
keys = [
    Overall cost structure, Capacity, RAW MATERIALS,
    BY-PRODUCT CREDITS, UTILITIES, PLANT GATE COST,
    PROCESS DESCRIPTION, AT 50% CAPACITY, PRODUCTION COSTS,
    INVESTMENT, US$ MILLION, PRODUCTION COSTS, US ¢/LB,
    VARIABLE COSTS, PRODUCTION COSTS, MAINTENANCE MATERIALS
  ]

So if you request keys you will get:

<class 'list'>: ['Overall cost structure', 'Capacity', 'RAW MATERIALS', 'BY-PRODUCT CREDITS', 'UTILITIES', 'PLANT GATE COST', 'PROCESS DESCRIPTION', 'AT 50% CAPACITY', 'PRODUCTION COSTS', 'INVESTMENT', 'US$ MILLION', 'PRODUCTION COSTS', 'US ¢/LB', 'VARIABLE COSTS', 'PRODUCTION COSTS', 'MAINTENANCE MATERIALS']

Code:

class AdvancedInterpolator(Interpolation):
    def before_get(self, parser, section, option, value, defaults):
        is_list = re.search(parser.LIST_MATCHER, value)
        if is_list:
            return parser.getlist(section, option, raw=True)
        return value


class AdvancedConfigParser(ConfigParser):

    _DEFAULT_INTERPOLATION = AdvancedInterpolator()

    LIST_SPLITTER = '\s*,\s*'
    LIST_MATCHER = '^\[([\s\S]*)\]$'

    def _to_list(self, str):
        is_list = re.search(self.LIST_MATCHER, str)
        if is_list:
            return re.split(self.LIST_SPLITTER, is_list.group(1))
        else:
            return re.split(self.LIST_SPLITTER, str)


    def getlist(self, section, option, conv=lambda x:x.strip(), *, raw=False, vars=None,
                  fallback=_UNSET, **kwargs):
        return self._get_conv(
                section, option,
                lambda value: [conv(x) for x in self._to_list(value)],
                raw=raw,
                vars=vars,
                fallback=fallback,
                **kwargs
        )

    def getlistint(self, section, option, *, raw=False, vars=None,
            fallback=_UNSET, **kwargs):
        return self.getlist(section, option, int, raw=raw, vars=vars,
                fallback=fallback, **kwargs)

    def getlistfloat(self, section, option, *, raw=False, vars=None,
            fallback=_UNSET, **kwargs):
        return self.getlist(section, option, float, raw=raw, vars=vars,
                fallback=fallback, **kwargs)

    def getlistboolean(self, section, option, *, raw=False, vars=None,
            fallback=_UNSET, **kwargs):
        return self.getlist(section, option, self._convert_to_boolean,
                raw=raw, vars=vars, fallback=fallback, **kwargs)

Ps keep in mind importance of indentdation. As reads in ConfigParser doc string:

Values can span multiple lines, as long as they are indented deeper than the first line of the value. Depending on the parser's mode, blank lines may be treated as parts of multiline values or ignored.

0

you can use list in config file then parse it in python

from ast import literal_eval

literal_eval("[1,2,3,4]")

import json

json.loads("[1,2,3,4]")

and also you can use json file behind your config file like this:

your config file :
[A]
json_dis = .example.jason
--------------------
your code :
import configparser
config = configparser.ConfigParser()
config.read('config.ini')
# getting items of section A
config.items('A')
# result is a list of key-values
0

An improvement on split(',') might be to treat the comma separated values as a record in a CSV file

import csv
my_list = list(csv.reader([config['Section 3']['barList']], dialect=csv.excel))[0]

You can configure a dialect to parse whatever style of CSV you like.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.