46

I would like to use a dictionary to insert values into a table, how would I do this?

import sqlite3

db = sqlite3.connect('local.db')
cur = db.cursor()

cur.execute('DROP TABLE IF EXISTS Media')

cur.execute('''CREATE TABLE IF NOT EXISTS Media(
                id INTEGER PRIMARY KEY, title TEXT, 
                type TEXT,  genre TEXT,
                onchapter INTEGER,  chapters INTEGER,
                status TEXT
                )''')


values = {'title':'jack', 'type':None, 'genre':'Action', 'onchapter':None,'chapters':6,'status':'Ongoing'}

#What would I Replace x with to allow a 
#dictionary to connect to the values? 
cur.execute('INSERT INTO Media VALUES (NULL, x)'), values)
cur.execute('SELECT * FROM Media')

meida = cur.fetchone()

print meida
0

9 Answers 9

51

If you're trying to use a dict to specify both the column names and the values, you can't do that, at least not directly.

That's really inherent in SQL. If you don't specify the list of column names, you have to specify them in CREATE TABLE order—which you can't do with a dict, because a dict has no order. If you really wanted to, of course, you could use a collections.OrderedDict, make sure it's in the right order, and then just pass values.values(). But at that point, why not just have a list (or tuple) in the first place? If you're absolutely sure you've got all the values, in the right order, and you want to refer to them by order rather than by name, what you have is a list, not a dict.

And there's no way to bind column names (or table names, etc.) in SQL, just values.

You can, of course, generate the SQL statement dynamically. For example:

columns = ', '.join(values.keys())
placeholders = ', '.join('?' * len(values))
sql = 'INSERT INTO Media ({}) VALUES ({})'.format(columns, placeholders)
values = [int(x) if isinstance(x, bool) else x for x in values.values()]
cur.execute(sql, values)

However, this is almost always a bad idea. This really isn't much better than generating and execing dynamic Python code. And you've just lost all of the benefits of using placeholders in the first place—primarily protection from SQL injection attacks, but also less important things like faster compilation, better caching, etc. within the DB engine.

It's probably better to step back and look at this problem from a higher level. For example, maybe you didn't really want a static list of properties, but rather a name-value MediaProperties table? Or, alternatively, maybe you want some kind of document-based storage (whether that's a high-powered nosql system, or just a bunch of JSON or YAML objects stored in a shelve)?


An alternative using named placeholders:

columns = ', '.join(my_dict.keys())
placeholders = ':'+', :'.join(my_dict.keys())
query = 'INSERT INTO my_table (%s) VALUES (%s)' % (columns, placeholders)
print query
cur.execute(query, my_dict)
con.commit()
6
  • 1
    @oche: "Not tested but the code seems wrong. Inserting keys into the table twice (for the placeholder) looks like a slight mistake." No, the execute call replaces each named placeholder by looking up its name as a key in my_dict. Therefore, it definitely should be keys, not values. Imagine that my_dict = {'spam': 'eggs'}. If you put a placeholder :eggs in the SQL statement, that wouldn't be found in my_dict; you need :spam.
    – abarnert
    Commented May 20, 2015 at 21:26
  • Placeholders automatically do correct escaping, whereas if you build the query yourself, you need to code up the special character escape logic.
    – snakehiss
    Commented Oct 7, 2015 at 20:24
  • 1
    Placeholders don't do correct escaping, even automatically. They simply don't do escaping, because they're not textually inserted into a SQL statement. Commented Feb 17, 2016 at 6:16
  • @JürgenA.Erhard Technically, under the covers, some SQL libraries actually do handle placeholders by just escaping them into the string… But users don't have to know that; each SQL library does the best thing with placeholders for its backend, and does it correctly, so +1 on your comment.
    – abarnert
    Commented Mar 9, 2016 at 9:20
  • 1
    because a dict has no order... as of Python 3.7 a dict now has an order. https://mail.python.org/pipermail/python-dev/2017-December/151283.html
    – user136036
    Commented Dec 31, 2019 at 20:36
24

There is a solution for using dictionaries. First, the SQL statement

INSERT INTO Media VALUES (NULL, 'x');

would not work, as it assumes you are referring to all columns, in the order they are defined in the CREATE TABLE statement, as abarnert stated. (See SQLite INSERT.)

Once you have fixed it by specifying the columns, you can use named placeholders to insert data. The advantage of this is that is safely escapes key-characters, so you do not have to worry. From the Python sqlite-documentation:

values = {
    'title':'jack', 'type':None, 'genre':'Action', 
    'onchapter':None,'chapters':6,'status':'Ongoing'
}
cur.execute(
    'INSERT INTO Media (id, title, type, onchapter, chapters, status)
     VALUES (:id, :title, :type, :onchapter, :chapters, :status);', 
    values
)
0
12

You could use named parameters:

cur.execute('INSERT INTO Media VALUES (NULL, :title, :type, :genre, :onchapter, :chapters, :status)', values)

This still depends on the column order in the INSERT statement (those : are only used as keys in the values dict) but it at least gets away from having to order the values on the python side, plus you can have other things in values that are ignored here; if you're pulling what's in the dict apart to store it in multiple tables, that can be useful.

If you still want to avoid duplicating the names, you could extract them from an sqlite3.Row result object, or from cur.description, after doing a dummy query; it may be saner to keep them around in python form near wherever you do your CREATE TABLE.

2

Here's a more generic way with the benefit of escaping:

# One way. If keys can be corrupted don't use.
sql = 'INSERT INTO demo ({}) VALUES ({})'.format(
            ','.join(my_dict.keys()),
            ','.join(['?']*len(my_dict)))

# Another, better way. Hardcoded w/ your keys.
sql = 'INSERT INTO demo ({}) VALUES ({})'.format(
            ','.join(my_keys),
            ','.join(['?']*len(my_dict)))

cur.execute(sql, tuple(my_dict.values()))
1
key_lst = ('status', 'title', 'chapters', 'onchapter', 'genre', 'type')
cur.execute('INSERT INTO Media (status,title,chapters,onchapter,genre,type) VALUES ' + 
            '(?,?,?,?,?,?);)',tuple(values[k] for k in key_lst))

Do your escaping right.

You probably also need a commit call in there someplace.

2
  • Why not use the key_lst to specify the column order instead of retyping it? cur.execute('INSERT INTO Media ({}) VALUES ();'.format(','.join(key_lst), ','.join(['?'] * len(key_lst))), tuple(values[k] for k in key_lst))
    – MrGumble
    Commented May 22, 2013 at 17:45
  • @MrGumble because you are still letting possibly user supplied data go into the sql statement with out escaping it. And I think you are missing a {} in the format statement. You should include this suggestion in your answer if you like it.
    – tacaswell
    Commented May 22, 2013 at 18:11
1

Super late to this, but figured I would add my own answer. Not an expert, but something I found that works.

There are issues with preserving order when using a dictionary, which other users have stated, but you could do the following:

# We're going to use a list of dictionaries, since that's what I'm having to use in my problem
input_list = [{'a' : 1 , 'b' : 2 , 'c' : 3} , {'a' : 14 , 'b' : '' , 'c' : 43}]
for i in input_list:
    # I recommend putting this inside a function, this way if this 
    # Evaluates to None at the end of the loop, you can exit without doing an insert
    if i :
        input_dict = i 
    else:
        input_dict = None
        continue
# I am noting here that in my case, I know all columns will exist.
# If you're not sure, you'll have to get all possible columns first.

keylist = list(input_dict.keys())
vallist = list(input_dict.values())

query = 'INSERT INTO example (' +','.join( ['[' + i + ']' for i in keylist]) + ') VALUES (' + ','.join(['?' for i in vallist]) + ')'

items_to_insert = list(tuple(x.get(i , '') for i in keylist) for x in input_list)
# Making sure to preserve insert order. 

conn = sqlite3.connect(':memory:')
cur = conn.cursor()
cur.executemany(query , items_to_insert)
conn.commit()
1
dictionary = {'id':123, 'name': 'Abc', 'address':'xyz'}    
query = "insert into table_name " + str(tuple(dictionary.keys())) + " values" + str(tuple(dictionary.values())) + ";"
cursor.execute(query)

query becomes

insert into table_name ('id', 'name', 'address') values(123, 'Abc', 'xyz');
0

I was having the similar problem so I created a string first and then passed that string to execute command. It does take longer time to execute but mapping was perfect for me. Just a work around:

create_string = "INSERT INTO datapath_rtg( Sr_no"
for key in record_tab:
    create_string = create_string+ " ," + str(key)
create_string = create_string+ ") VALUES("+ str(Sr_no) 
for key in record_tab:
    create_string = create_string+ " ," + str(record_tab[key])
create_string = create_string + ")"
cursor.execute(create_string)

By doing above thing I ensured that if my dict (record_tab) doesn't contain a particular field then the script wont throw out error and proper mapping can be done which is why I used dictionary at the first place.

0

I was having a similar problem and ended up with something not entirely unlike the following (Note - this is the OP's code with bits changed so that it works in the way they requested)-

import sqlite3
db = sqlite3.connect('local.db')
cur = db.cursor()

cur.execute('DROP TABLE IF EXISTS Media')

cur.execute('''CREATE TABLE IF NOT EXISTS Media(
                id INTEGER PRIMARY KEY, title TEXT, 
                type TEXT,  genre TEXT,
                onchapter INTEGER,  chapters INTEGER,
                status TEXT
                )''')


values = {'title':'jack', 'type':None, 'genre':'Action',     'onchapter':None,'chapters':6,'status':'Ongoing'}

#What would I Replace x with to allow a 
#dictionary to connect to the values? 
#cur.execute('INSERT INTO Media VALUES (NULL, x)'), values)
# Added code.
cur.execute('SELECT * FROM Media')
colnames = cur.description
list = [row[0] for row in cur.description]
new_list = [values[i] for i in list if i in values.keys()]
sql = "INSERT INTO Media VALUES ( NULL, "
qmarks = ', '.join('?' * len(values))
sql += qmarks + ")"
cur.execute(sql, new_list)
#db.commit() #<-Might be important.
cur.execute('SELECT * FROM Media')
media = cur.fetchone()
print (media)

Not the answer you're looking for? Browse other questions tagged or ask your own question.