18

I am used to (spoiled by?) python's SQLite interface to deal with SQL databases. One nice feature in python's SQLite's API the "context manager," i.e., python's with statement. I usually execute queries in the following way:

import as sqlite

with sqlite.connect(db_filename) as conn:
    query = "INSERT OR IGNORE INTO shapes VALUES (?,?);"
    results = conn.execute(query, ("ID1","triangle"))

With the code above, if my query modifies the database and I forget to run conn.commit(),the context manager runs it for me automatically upon exiting the with statement. It also handles exceptions nicely: if an exception occurs before I commit anything, then the database is rolled back.

I am now using the MySQLdb interface, which doesn't seem to support a similar context manager out of the box. How do I create my own? There is a related question here, but it doesn't offer a complete solution.

0

2 Answers 2

28

Previously, MySQLdb connections were context managers. As of this commit on 2018-12-04, however, MySQLdb connections are no longer context managers, and users must explicitly call conn.commit() or conn.rollback(), or write their own context manager, such as the one below.


You could use something like this:

import config
import MySQLdb
import MySQLdb.cursors as mc
import _mysql_exceptions
import contextlib
DictCursor = mc.DictCursor
SSCursor = mc.SSCursor
SSDictCursor = mc.SSDictCursor
Cursor = mc.Cursor

@contextlib.contextmanager
def connection(cursorclass=Cursor,
               host=config.HOST, user=config.USER,
               passwd=config.PASS, dbname=config.MYDB,
               driver=MySQLdb):
    connection = driver.connect(
            host=host, user=user, passwd=passwd, db=dbname,
            cursorclass=cursorclass)
    try:
        yield connection
    except Exception:
        connection.rollback()
        raise
    else:
        connection.commit()
    finally:
        connection.close()

@contextlib.contextmanager
def cursor(cursorclass=Cursor, host=config.HOST, user=config.USER,
           passwd=config.PASS, dbname=config.MYDB):
    with connection(cursorclass, host, user, passwd, dbname) as conn:
        cursor = conn.cursor()
        try:
            yield cursor
        finally:
            cursor.close()


with cursor(SSDictCursor) as cur:
    print(cur)
    connection = cur.connection
    print(connection)
    sql = 'select * from table'
    cur.execute(sql)
    for row in cur:
        print(row)

To use it you would place config.py in your PYTHONPATH and define the HOST, USER, PASS, MYDB variables there.

6
  • Excellent solution! Not only did you give the answer for MySQLdb, it can also be used with other drivers. Also, oursql looks promising. Thanks.
    – conradlee
    Commented Nov 10, 2011 at 14:45
  • @unutbu shouldn't the connectio be initialized in the enter function? I think it's much safer. Here's an example geeksforgeeks.org/context-manager-in-python
    – callmeGuy
    Commented Apr 4, 2019 at 9:03
  • @callmeGuy: The __exit__ function assumes that self.connection exists. If you instantiate the connection in the __enter__ function and it raises an exception, then the __exit__ function will raise a second exception since there would be no connection to call rollback or commit on. So no, it would not be safer.
    – unutbu
    Commented Apr 4, 2019 at 11:24
  • @callmeGuy: Further corroboration can be found in the github source code. You can see the developers (prior to the change) did not initialize the connection in __enter__. See also this doc example.
    – unutbu
    Commented Apr 4, 2019 at 11:25
  • @callmeGuy: Your question did prompt me to rewrite the code, however. Connections and cursors are not the same things so I've separated the two so each is its own context manager. I also switched to using the contextlib.contextmanager decorator instead of defining __enter__ and __exit__ explicitly since it makes the code shorter and more readable.
    – unutbu
    Commented Apr 4, 2019 at 11:38
20

Think things have changed since this question was originally asked. Somewhat confusingly (from my point of view at least), for recent versions of MySQLdb, if you use a connection in a context you get a cursor (as per the oursql example), not something that closes automatically (as you would if you opened a file for instance).

Here's what I do:

from contextlib import closing
with closing(getConnection()) as conn: #ensure that the connection is closed
    with conn as cursor:               #cursor will now auto-commit
        cursor.execute('SELECT * FROM tablename')
1
  • 2
    Auto commit? As in, auto commit mode, so you have no transaction?
    – Halfgaar
    Commented Feb 28, 2018 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.