112

I'm not asking for the SHOW COLUMNS command.

I want to create an application that works similarly to heidisql, where you can specify an SQL query and when executed, returns a result set with rows and columns representing your query result. The column names in the result set should match your selected columns as defined in your SQL query.

In my Python program (using MySQLdb) my query returns only the row and column results, but not the column names. In the following example the column names would be ext, totalsize, and filecount. The SQL would eventually be external from the program.

The only way I can figure to make this work, is to write my own SQL parser logic to extract the selected column names.

Is there an easy way to get the column names for the provided SQL? Next I'll need to know how many columns does the query return?

# Python

import MySQLdb

#===================================================================
# connect to mysql
#===================================================================

try:
    db = MySQLdb.connect(host="myhost", user="myuser", passwd="mypass",db="mydb")
except MySQLdb.Error, e:
    print "Error %d: %s" % (e.args[0], e.args[1])
    sys.exit (1)

#===================================================================
# query select from table
#===================================================================

cursor = db.cursor ()   

cursor.execute ("""\
     select ext,
        sum(size) as totalsize,
        count(*) as filecount
     from fileindex
    group by ext
    order by totalsize desc;
""")

while (1):
    row = cursor.fetchone ()
    if row == None:
        break
    print "%s %s %s\n" % (row[0], row[1], row[2])

cursor.close()
db.close()      
1
  • 1
    cursor.column_names
    – Amin
    May 21, 2019 at 14:52

13 Answers 13

278

cursor.description will give you a tuple of tuples where [0] for each is the column header.

num_fields = len(cursor.description)
field_names = [i[0] for i in cursor.description]
5
  • 12
    cursor.column_names does it too
    – Amin
    May 21, 2019 at 14:54
  • 2
    In MySQLdb 1.2.5 cursor.column_names is missing. Mar 25, 2020 at 19:20
  • @YaroslavNikitenko .column_names is gone? I need to use .description to get the column names?
    – SteveS
    Aug 20, 2021 at 8:26
  • @SteveS from what I wrote last year it seems that :) Sep 8, 2021 at 14:11
  • To make it shorter: field_names = next(zip(*cursor.description))
    – dibery
    Apr 28 at 7:02
46

This is the same as thefreeman but more in pythonic way using list and dictionary comprehension

columns = cursor.description 
result = [{columns[index][0]:column for index, column in enumerate(value)} for value in cursor.fetchall()]

pprint.pprint(result)
34

Similar to @James answer, a more pythonic way can be:

fields = [field_md[0] for field_md in cursor.description]
result = [dict(zip(fields,row)) for row in cursor.fetchall()]

You can get a single column with list comprehension over the result:

extensions = [row['ext'] for row in result)

or filter results using an additional if in the list comprehension:

large = [row for row in result if row['filesize'] > 1024 and row['filesize'] < 4096]

or accumulate values for filtered columns:

totalTxtSize = reduce(
        lambda x,y: x+y,
        filter(lambda x: x['ext'].lower() == 'txt', result)
)
4
  • Thorough. Is there an advantage to using fetchall over iterating on the cursor?
    – cs_alumnus
    Sep 24, 2014 at 17:33
  • @kzarns: I'd say that making the example shorter, and getting a full list. You might create a generator function instead that iterates over the cursor, using yield, and might use less memory.
    – juandesant
    Sep 26, 2014 at 15:32
  • 4
    Doesn't work with me, only the first row is converterted, slightly adaption to make it work: fields = [i[0] for i in cursor.description] then result = [dict(zip(fields,row)) for row in cursor.fetchall()] wil work
    – pdem
    Nov 6, 2020 at 10:25
  • 1
    Very nice. You could also create a DataFrame from result with pd.DataFrame.from_records and do all that filtering and aggregation in pandas. Although, if you are up to using pandas, it might be easier to use sqlalchemy and pd.read_sql in the first place.
    – Patrick H.
    Aug 12, 2022 at 7:39
14

I think this should do what you need (builds on the answer above) . I am sure theres a more pythony way to write it, but you should get the general idea.

cursor.execute(query)
columns = cursor.description
result = []
for value in cursor.fetchall():
    tmp = {}
    for (index,column) in enumerate(value):
        tmp[columns[index][0]] = column
    result.append(tmp)
pprint.pprint(result)
1
  • 1
    is there any other simple way, why it does not return value with fields? Jul 28, 2018 at 10:42
9

You could also use MySQLdb.cursors.DictCursor. This turns your result set into a python list of python dictionaries, although it uses a special cursor, thus technically less portable than the accepted answer. Not sure about speed. Here's the edited original code that uses this.

#!/usr/bin/python -u

import MySQLdb
import MySQLdb.cursors

#===================================================================
# connect to mysql
#===================================================================

try:
    db = MySQLdb.connect(host='myhost', user='myuser', passwd='mypass', db='mydb', cursorclass=MySQLdb.cursors.DictCursor)
except MySQLdb.Error, e:
    print 'Error %d: %s' % (e.args[0], e.args[1])
    sys.exit(1)

#===================================================================
# query select from table
#===================================================================

cursor = db.cursor()

sql = 'SELECT ext, SUM(size) AS totalsize, COUNT(*) AS filecount FROM fileindex GROUP BY ext ORDER BY totalsize DESC;'

cursor.execute(sql)
all_rows = cursor.fetchall()

print len(all_rows) # How many rows are returned.
for row in all_rows: # While loops always make me shudder!
    print '%s %s %s\n' % (row['ext'], row['totalsize'], row['filecount'])

cursor.close()
db.close()  

Standard dictionary functions apply, for example, len(row[0]) to count the number of columns for the first row, list(row[0]) for a list of column names (for the first row), etc. Hope this helps!

2
9

This is only an add-on to the accepted answer:

def get_results(db_cursor):
    desc = [d[0] for d in db_cursor.description]
    results = [dotdict(dict(zip(desc, res))) for res in db_cursor.fetchall()]
    return results

where dotdict is:

class dotdict(dict):
    __getattr__ = dict.get
    __setattr__ = dict.__setitem__
    __delattr__ = dict.__delitem__

This will allow you to access much easier the values by column names.
Suppose you have a user table with columns name and email:

cursor.execute('select * from users')
results = get_results(cursor)
for res in results:
  print(res.name, res.email)
0
5

Something similar to the proposed solutions, only the result is json with column_header : vaule for db_query ie sql.

cur = conn.cursor()
cur.execute(sql)
res = [dict((cur.description[i][0], value) for i, value in enumerate(row)) for row in cur.fetchall()]

output json example:

[
   {
      "FIRST_ROW":"Test 11",
      "SECOND_ROW":"Test 12",
      "THIRD_ROW":"Test 13"
   },
   {
      "FIRST_ROW":"Test 21",
      "SECOND_ROW":"Test 22",
      "THIRD_ROW":"Test 23"
   }
]
3

Try:

cursor.column_names

mysql connector version:

mysql.connector.__version__
'2.2.9'
1
  • maybe it works for MySQL, but didn't work for sqlite for me.
    – alex
    Nov 2, 2022 at 12:02
2

Looks like MySQLdb doesn't actually provide a translation for that API call. The relevant C API call is mysql_fetch_fields, and there is no MySQLdb translation for that

1

You can also do this to just get the field titles:

table = cursor.description
check = 0
for fields in table:
    for name in fields:
        if check < 1:
            print(name),
        check +=1
    check =0
1
  • Very nice idea thanks! I really wanted to get the columns as a result of a query, and your suggestion provides a nice way <cursor>.execute('describe <table>') and then fetch
    – Cryo
    Apr 14, 2020 at 20:34
1

cursor.column_names is a nice and simple one.

-1

column_names = cursor.field_names

1
-1

found an easy way of having colums like sql using pymysql and pandas

import pymysql
import pandas as pd

db  = pymysql.connect(host="myhost", user="myuser", passwd="mypass", db="mydb")

query  = """SELECT ext
              SUM(size) as totalsize,
              COUNT(*) as filecount
            FROM fileindex
            GROUP BY ext 
            ORDER BY totalsize DESC;
         """
df = pd.read_sql_query(query,db)

the DataFrame will have column names ext,totalsize,filecount by default no need to do additional stuff.

for example in my case: enter image description here

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.