1

When I try and execute the code below, python gives me a mariadb.ProgrammingError: Cursor is closed even though the data was successfully written to the db. I guess there is something wrong with returning the rowcount (see the full traceback below). Is this a possible bug or am I missing something?

code sample

import pandas as pd
from sqlalchemy import create_engine
conn_str = 'mariadb+mariadbconnector://user:password@localhost:3306/database_name'
engine = create_engine(conn_str)
df = pd.DataFrame(data={'col1': [1, 2, 3], 'col2': ['A', 'B', 'C']})
df.to_sql('pandas_test_table', index=False, con=engine, if_exists='replace')

traceback

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pandas\core\generic.py", line 2951, in to_sql
    return sql.to_sql(
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pandas\io\sql.py", line 697, in to_sql
    return pandas_sql.to_sql(
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pandas\io\sql.py", line 1739, in to_sql
    total_inserted = sql_engine.insert_records(
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pandas\io\sql.py", line 1322, in insert_records
    return table.insert(chunksize=chunksize, method=method)
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pandas\io\sql.py", line 950, in insert
    num_inserted = exec_insert(conn, keys, chunk_iter)
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pandas\io\sql.py", line 858, in _execute_insert
    return result.rowcount
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\util\langhelpers.py", line 1113, in __get__
    obj.__dict__[self.__name__] = result = self.fget(obj)
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\engine\cursor.py", line 1691, in rowcount
    self.cursor_strategy.handle_exception(self, self.cursor, e)
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\engine\cursor.py", line 846, in handle_exception
    raise err
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\engine\cursor.py", line 1689, in rowcount
    return self.context.rowcount
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\engine\default.py", line 1439, in rowcount
    return self.cursor.rowcount
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\mariadb\cursors.py", line 541, in rowcount
    self.check_closed()
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\mariadb\cursors.py", line 55, in check_closed
    raise mariadb.ProgrammingError("Cursor is closed")
mariadb.ProgrammingError: Cursor is closed

package version

mariadb==1.1.4
pandas==1.4.3
SQLAlchemy==1.4.40

4 Answers 4

8

I had attempted Georg Richter's recommendation regarding installing MariaDB 1.1.5 from source or rolling back to 1.1.3 and was still receiving the error message.

Eventually, I came across this documentation and had changed the engine DBAPI from

uri = 'mariadb+mariadbconnector://Username:Password@localhost/test'

to

uri = 'mariadb+pymysql://Username:Password@localhost/test'

And it worked.

1
  • this works perfectly on 11 Mar 2023
    – yts61
    Mar 11, 2023 at 10:24
1

2023-10-29 Update

For anyone that lands here from a Google search, and is in a position where they can't/don't want to change from mariadbconnector to pymysql, this bug should be fixed in:

  • mariadb==1.1.8 (bug ticket here)
  • SQLAlchemy==2.0.22 (bug ticket here)
0

It's a bug in MariaDB Connector/Python.

It's already fixed in 1.1.5 (not released yet). If you don't want to downgrade to 1.1.3 you have to build 1.1.5 from source.

Update from 2023-09-27: After investigating bug CONPY-269 it seems the problem is in SQL Alchemy/Pandas: After closing the cursor with cursor.close() the property cursor.rowcount is accessed. This is in contradiction to PEP-249:

.close() Close the cursor now (rather than whenever del is called). The cursor will be unusable from this point forward; an Error (or subclass) exception will be raised if any operation is attempted with the cursor.

1
  • using 1.1.6 and still getting the same error. It has not been fixed as far as I can see.
    – maximus
    Jun 26, 2023 at 21:52
0

Hummm... I'm using mariadb version 1.1.5.post3, but I get the same error. Just the last lines of the data insertion code, which doesn't raise any error in case I use the pymysql connector:

new_employee = {'id': '105', 'first_name': 'Joao',
                'last_name': 'da Silva', 'description': 'Lorem Ipsum', 'active': '1'}
emp = {k: [v] for k, v in new_employee.items()}
df = pd.DataFrame(
    emp, columns=['id', 'first_name', 'last_name', 'description', 'active'])
df.to_sql('employees', con=cnx, index=False, if_exists='append')

The error:

    raise mariadb.ProgrammingError("Cursor is closed")
mariadb.ProgrammingError: Cursor is closed

Just in case, I've tried to downgrade to the version 1.1.5 mentioned above (using pip install, not from the source). Other packages versions:

SQLAlchemy        1.4.46
pandas            1.5.2

P. S. Although we have this error, the data is inserted into the Maria DB table.

5
  • This does not really answer the question. If you have a different question, you can ask it by clicking Ask Question. To get notified when this question gets new answers, you can follow this question. Once you have enough reputation, you can also add a bounty to draw more attention to this question. - From Review
    – Jeremie
    Jan 26, 2023 at 11:23
  • Hi, @Jeremie When I created the comment above, I thought I was replying to Georg Richter. But I noticed after I made the comment to the general question in the topic. Should I just delete it? I think's an additional information to the answer that used the lasted MariaDB driver, but that doesn't work.
    – everton137
    Jan 27, 2023 at 15:52
  • when responding to someone, you shall click the "add a comment" link following their post, not replying to the main question.
    – Jeremie
    Jan 27, 2023 at 18:10
  • Yes, I've learned after. I am not sure if I should simply delete the mistake or not.
    – everton137
    Jan 28, 2023 at 20:21
  • Okay, I just noticed I cannot comment a comment yet. (Except my own, I suppose)
    – everton137
    Jan 28, 2023 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.