81

I already have an existing Database that has a lot of tables and a lot of data in MySQL. I intend to create a Flask app and use sqlalchemy along with it. Now I asked out on irc and looked around on google and tried the following ideas:

First I used sqlacodegen to generate the models from my DB. But then I was confused about it a little and looked some more. And I found this.

This looked like an elegant solution.

So Second, I rewrote my models.py according to the solution there and now I am even more confused. I am looking for the best approach to build this flask app along with the already existing DB.

I looked into the flask documentation but didnt really get any help for a project with an already existing db. There is a lot of good stuff for creating something from scratch, creating the db and all. But I am really confused.

Please Note that its my first day with Flask, but I have experience with Django, so the basic concepts are not a hurdle. I need some guidance in choosing the best approach for this usecase. A detailed explanation would be greatly appreciated. By detailed I definitely do not expect someone to write all the code and spoon feed me on this, but just enough to get me started, that is integrate this db seamlessly into flask via sqlalchemy. Note my DB is in MySQL.

10 Answers 10

109

I'd say your question has nothing to do with flask at all. For example, you don't have a problem with the templates, routes, views or logon decorators.

Where you struggle at is at SQLAlchemy.

So my suggestion is to ignore Flask for a while and get used to SQLAlchemy first. You need to get used to your existing database and how to access it from SQLAlchemy. Use some MySQL documentation tool to find your way around this. The start with something like this (note that it has nothing to do with Flask ask all ... yet):

#!/usr/bin/python
# -*- mode: python -*-

from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base

engine = create_engine('sqlite:///webmgmt.db', convert_unicode=True, echo=False)
Base = declarative_base()
Base.metadata.reflect(engine)


from sqlalchemy.orm import relationship, backref

class Users(Base):
    __table__ = Base.metadata.tables['users']


if __name__ == '__main__':
    from sqlalchemy.orm import scoped_session, sessionmaker, Query
    db_session = scoped_session(sessionmaker(bind=engine))
    for item in db_session.query(Users.id, Users.name):
        print item

In the line "engine =" you need to provide your path to your MySQL database, so that SQLAlchemy finds it. In my case I used a pre-existing sqlite3 database.

In the line "class Users(Base)" you need to use one of existing tables in your MySQL database. I knew that my sqlite3 database had a table named "users".

After this point, SQLalchemy knows how to connect to your MySQL database and it knows about one of the tables. You need now to add all the other tables that you care for. Finally, you need to specify relationships to SQLalchemy. Here I mean things like one-to-one, one-to-many, many-to-many, parent-child and so on. The SQLAlchemy web site contains a rather lenghty section about this.

After the line "if __name__ == '__main__'" just comes some test code. It will be executed if I don't import my python script, but run. Here you see that I create a DB session and is that for a very simply query.

My suggestion is that you first read about the important parts of SQLAlchemy's documentation, for example the descriptive table definition, the relationship model and how to query. Once you know this, you can change the last part of my example into a controller (e.g. using Python's yield method) and write a view that uses that controller.

4
  • Many thanks for the descriptive answer. I have been reading into sqlalchemy documentation and specifically the relationships part. I have implemented your solution so far, and your answer seems to give me a push from the point where I felt stuck, but I am wondering, that how should I integrate that into flask? I mean in flask docs there was a lot about init_db() which was specifically to create a database. What about this case? Thanks agian. :) Jul 15, 2013 at 12:33
  • And then when you want to get this going in Flask: flask.pocoo.org/docs/patterns/sqlalchemy/#declarative Apr 5, 2014 at 0:39
  • 1
    The sqlalchemy docs have a section on how to configure the engine paths for a variety of different databases: docs.sqlalchemy.org/en/rel_0_9/core/engines.html, also the engine path can be set up with keywords instead of as a single string using the URL option: docs.sqlalchemy.org/en/rel_0_9/core/…, my passwords have special characters and this was helpful
    – sequoia
    Mar 21, 2015 at 2:26
  • one of the most useful comments that exists on StackOverflow
    – ccdrm
    Mar 26, 2018 at 22:19
68

The key to connecting Holger's answer to a flask context is that db.Model is a declarative_base object like Base. Took me a while to notice this important sentence in flask-sqlalchemy's documentation

Below are the steps I used for my app:

  1. initiate a db object in the usual flask-alchemy manner:db = SQLAlchemy(app). Note you'll need to set app.config['SQLALCHEMY_DATABASE_URI'] = 'connection_string' before that.

  2. bind the declarative base to an engine: db.Model.metadata.reflect(db.engine)

  3. Then you can use existing tables easily (eg. I have a table called BUILDINGS):

    class Buildings(db.Model):
        __table__ = db.Model.metadata.tables['BUILDING']
    
        def __repr__(self):
            return self.DISTRICT
    

Now your Buildings class will follow the existing schema. You can try dir(Buildings) in a Python shell and see all the columns already listed.

4
  • 6
    The insight that 'db.Model is a declarative_base object like Base' just saved my day. Until i read this i had no clue to do this in flask-alchemy. Thank you. Apr 19, 2016 at 8:23
  • 5
    when it comes to Flask-sqlalchemy db.Model is analogous to Base,. db.session is analogous to session. Apr 23, 2016 at 11:26
  • 1
    Please write blog on it, there are may misleading answers on Net, this is missing everywhere else Mar 18, 2020 at 9:42
  • how do i perform a query (select all entries) on this model? Buildings.query.filter_by(b_id=1).first() gives an error -> AttributeError: 'function' object has no attribute 'query' Apr 26, 2021 at 6:12
21

I recently went through the same thing, with the additional challenge of linking the models across two databases.

I used Flask-SQLAlchemy and all I had to do was define my models in the same way as my database tables looked. What I found difficult was figuring out exactly what my project structure should look like.

My project was a Restful API, and this is what I ended up with:

conf/
    __init__.py
    local.py
    dev.py
    stage.py
    live.py
deploy/
    #nginx, uwsgi config, etc
middleware/
    authentication.py
app_name/
    blueprints/
        __init__.py
        model_name.py #routes for model_name
        ...
    models/
        __init.py
        model_name.py
    __init__.py
    database.py
tests/
    unit/
        test_etc.py
        ...
run.py

Files of note:

conf/xxx.py

This is how we tell Flask-SQLAlchemy what to connect to, plus you can put any other config items in here (like log location, debugging config, etc).

SQLALCHEMY_DATABASE_URI = 'mysql://username:password@host:port/db_name'

app_name/__init__.py

This is where I create my app and initialise the db. This db object will be imported and used across the entire app (i.e., in the models, tests, etc). I also set my logger, initialise my APIs and blueprints and attach my middleware in here (not shown).

from app_name.database import db
from flask import Flask

def create_app(*args, **kwargs):
    env = kwargs['env']
    app = Flask(__name__)
    app.config.from_object('conf.%s' % env)
    db.init_app(app)
    return app

app_name/database.py

from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy()

app_name/models/model_name.py

from services.database import db


class Bar(db.Model):

    __tablename__ = 'your_MySQL_table_name'

    id = db.Column('YourMySQLColumnName', db.Integer, primary_key=True)
    name = db.Column('WhateverName', db.String(100))
    foo = db.Column(db.ForeignKey('another_MySQLTableName.id'))

class Foo(db.Model):

    __tablename__ = 'another_MySQLTableName'

    id = db.Column('FooId', db.Integer, primary_key=True)
    ...

run.py

#! /usr/bin/env python

from app_name import create_app

app = create_app(env='local')

if __name__ == '__main__':
    app.run()

I use run.py to run the app locally, but I use nginx + uWSGI to run the app in the dev/stage/live environments.

I'm guessing you'll have a views/ directory in there in addition to this though.

4
  • why a models/ separately? Isnt models.py inside app/ sufficient? Jul 15, 2013 at 15:37
  • 1
    No, I have a lot of models, so I've split them into separate files. Jul 15, 2013 at 21:40
  • You mentioned your models covered more than one database but how this was handled is not immediately obvious to me as the above seems to only reflect one database object? Am I misunderstanding something?
    – W.Prins
    Aug 24, 2016 at 16:16
  • Can you show your middlewares?
    – brian_wang
    Feb 23, 2022 at 4:59
12

I think the easiest way to use an existing database with sqlalchemy is to use AutomapBase class . A sample code from docs is as follows:

from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import Session
from sqlalchemy import create_engine

Base = automap_base()

# engine, suppose it has two tables 'user' and 'address' set up
engine = create_engine("sqlite:///mydatabase.db")

# reflect the tables
Base.prepare(engine, reflect=True)

# mapped classes are now created with names by default
# matching that of the table name.
User = Base.classes.user
Address = Base.classes.address

session = Session(engine)

# rudimentary relationships are produced
session.add(Address(email_address="[email protected]", user=User(name="foo")))
session.commit()

# collection-based relationships are by default named
# "<classname>_collection"
print (u1.address_collection)

Refer SqlAlchemy-Automap for details and more complicated usages

8

I try to use autogenerated but nothing works or I couldn't run it. When I searching generate code using sqlacodegen I find https://github.com/ksindi/flask-sqlacodegen, you can generate the code just

flask-sqlacodegen  mysql://username:password@host:port/db_name --schema yourschema --tables table1,table2 --flask

I tried and it works perfectly

2
  • I am trying below connection string from terminal to connect to my local sql server instance flask-sqlacodegen --outfile models.py mssql://user:password@localhost/db_name I get error "Data source not found" and no default driver specified. I am unixodbc, FREETDS installed and able to connect programmatically and via isql, tsql commands. Apr 7, 2017 at 22:11
  • 1
    I want to thank you from the bottom of my heart for introducing me to this wonderful module. A life-saver when dealing with complex relationships in your databases.
    – Septem151
    Jul 13, 2020 at 20:40
1

This is an alternative way to set up the engine path described in Holger's answer. Convenient if there are special characters in your user name or password.

from sqlalchemy.engine.url import URL
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base

engine_URL = URL('mssql+pymssql',
                 username='DOMAIN\\USERNAME', 
                 password="""p@ssword'!""", 
                 host='host.com', 
                 database='database_name')

engine = create_engine(engine_URL)
Base = declarative_base()
Base.metadata.reflect(engine)
1

This solution worked for me

"""Example for reflecting database tables to ORM objects

This script creates classes for each table reflected
from the database.

Note: The class names are imported to the global namespace using
the same name as the tables. This is useful for quick utility scripts.
A better solution for production code would be to return a dict
of reflected ORM objects.
"""

from sqlalchemy import create_engine, MetaData
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base


def reflect_all_tables_to_declarative(uri):
"""Reflects all tables to declaratives

Given a valid engine URI and declarative_base base class
reflects all tables and imports them to the global namespace.

Returns a session object bound to the engine created.
"""

# create an unbound base our objects will inherit from
Base = declarative_base()

engine = create_engine(uri)
metadata = MetaData(bind=engine)
Base.metadata = metadata

g = globals()

metadata.reflect()

for tablename, tableobj in metadata.tables.items():
    g[tablename] = type(str(tablename), (Base,), {'__table__' : tableobj })
    print("Reflecting {0}".format(tablename))

Session = sessionmaker(bind=engine)
return Session()


# set to database credentials/host
CONNECTION_URI = "postgres://..."

session = reflect_all_tables_to_declarative(CONNECTION_URI)

# do something with the session and the orm objects
results = session.query(some_table_name).all()
0

I'd suggest using dynamic class

from flask import Flask
from sqlalchemy import Table, MetaData
from flask_sqlalchemy import SQLAlchemy
import os

app = Flask(__name__)

class Config(Object):
    basedir = os.path.abspath(os.path.dirname(__file__))
    SQLALCHEMY_DATABASE_URI = 'sqlite:///' +  + os.path.join(basedir, 'db.sqlite') 
    SQLALCHEMY_TRACK_MODIFICATIONS = False

app.config.from_object(config)
db = SQLAlchemy(app)
metadata = MetaData()

table_reflection = Table("table_name", metadata, autoload=True, autoload_with=db.engine)
attrs = {"__table__": table_reflection}
TableModel = type("table_name", (db.Model,), attrs)

Now, TableModel can be used as models class

# Querying
TableModel.query.all()
TableModel.query.get(1)
TableModel.query.filter_by(id=2).first()

# insertion
new_row = TableModel()
db.session.add(new_row)
db.session.commit()
1
  • This almost work for me, thanks a lot. but I get TypeError: string argument without an encoding error, I can't find where i can put encoding information here when i connect to table. Apr 25, 2021 at 15:46
-1

alembic (the tool behind flask-sqlalchemy) can be configured to ignore tables. The configuration isn't too hard to setup. see: https://gist.github.com/utek/6163250

1
  • 1
    I think you'll find that alembic works in the opposite direction to the question. Question wants to make models reflect database, alembic tries to make database reflect models. Also, alembic underscores Flask-Migrate library, not Flask-SQLAlchemy.
    – SuperShoot
    Sep 3, 2019 at 10:48
-1

If your application is simple you can use db.reflect(app) on your model file:

def configure(app):
    db.init_app(app)
    db.reflect(app=app)
    app.db = db
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.