I'm busy writing a small game server to try out flask. The game exposes an API via REST to users. It's easy for users to perform actions and query data, however I'd like to service the "game world"
outside the app.run()
loop to update game entities, etc. Given that Flask is so cleanly implemented, I'd like to see if there's a Flask way to do this.
-
You mean something like Flask-Admin? Or if you are using an ORM (SQL-Alchemy), then you can just create a new db session to query the database even if the application is running.– reptilicusJan 17, 2013 at 18:38
-
It looks like there's a hackish way to do it, but I don't think this is technically supported. I also found this answer, which talks about using flask-celery for this.– girasquidJan 17, 2013 at 18:42
-
If you actually need to do a lot of computation, you might want to use the subprocess module, and simply spawn new processes to do that additional computation.– MausJan 17, 2013 at 22:02
-
@girasquid Agreed, celery or some other task queue system is ideal for this sort of thing - you generally have less control over threads or sub-processes (since the parent process may be reaped by the server without notice).– Sean VieiraJan 18, 2013 at 3:14
-
That is a plan, however the sub process will be manipulating data structures, that you'd like to access and set via the exposed flask api. Will I not run into problems?– MarinusJan 18, 2013 at 14:41
3 Answers
Your additional threads must be initiated from the same app that is called by the WSGI server.
The example below creates a background thread that executes every 5 seconds and manipulates data structures that are also available to Flask routed functions.
import threading
import atexit
from flask import Flask
POOL_TIME = 5 #Seconds
# variables that are accessible from anywhere
commonDataStruct = {}
# lock to control access to variable
dataLock = threading.Lock()
# thread handler
yourThread = threading.Thread()
def create_app():
app = Flask(__name__)
def interrupt():
global yourThread
yourThread.cancel()
def doStuff():
global commonDataStruct
global yourThread
with dataLock:
pass
# Do your stuff with commonDataStruct Here
# Set the next thread to happen
yourThread = threading.Timer(POOL_TIME, doStuff, ())
yourThread.start()
def doStuffStart():
# Do initialisation stuff here
global yourThread
# Create your thread
yourThread = threading.Timer(POOL_TIME, doStuff, ())
yourThread.start()
# Initiate
doStuffStart()
# When you kill Flask (SIGTERM), clear the trigger for the next thread
atexit.register(interrupt)
return app
app = create_app()
Call it from Gunicorn with something like this:
gunicorn -b 0.0.0.0:5000 --log-config log.conf --pid=app.pid myfile:app
-
14I found this to be problematic when using flask's auto-reload functionality (a new thread got created on every reload). To fix this, I used werkzeug.serving.is_running_from_reloader to only create it when the app is not running from the reloader. Sep 6, 2015 at 13:05
-
2
-
This is a nice solution; helps deal with flask apps that use multiprocessing or threading modules. I like it. Jan 14, 2017 at 7:08
-
2This example is a little confusing because the object created called "yourThread" is not a thread. It's a timer: suggest you rename it. And, when yourTimer is executed ( in doStuff ), I don't know if yourThread is valid - ie, if you can execute cancel on a Timer that hasn't been executed. It has the efficiency issue that it's creating a new object every execution, if that might be an issue. Apr 10, 2017 at 4:07
-
1The correct statement for checking "is_running_in_background()" is like so: from werkzeug.serving import is_running_from_reloader if is_running_from_reloader() == False: startBackground() Apr 10, 2017 at 16:25
In addition to using pure threads or the Celery queue (note that flask-celery is no longer required), you could also have a look at flask-apscheduler:
https://github.com/viniciuschiele/flask-apscheduler
A simple example copied from https://github.com/viniciuschiele/flask-apscheduler/blob/master/examples/jobs.py:
from flask import Flask
from flask_apscheduler import APScheduler
class Config(object):
JOBS = [
{
'id': 'job1',
'func': 'jobs:job1',
'args': (1, 2),
'trigger': 'interval',
'seconds': 10
}
]
SCHEDULER_API_ENABLED = True
def job1(a, b):
print(str(a) + ' ' + str(b))
if __name__ == '__main__':
app = Flask(__name__)
app.config.from_object(Config())
scheduler = APScheduler()
# it is also possible to enable the API directly
# scheduler.api_enabled = True
scheduler.init_app(app)
scheduler.start()
app.run()
-
Can we start multiple jobs concurently that will not share same object storage?– alperSep 13, 2021 at 21:23
First, you should use any WebSocket or polling mechanics to notify the frontend part about changes that happened. I use Flask-SocketIO
wrapper, and very happy with async messaging for my tiny apps.
Nest, you can do all logic which you need in a separate thread(s), and notify the frontend via SocketIO
object (Flask holds continuous open connection with every frontend client).
As an example, I just implemented page reload on backend file modifications:
<!doctype html>
<script>
sio = io()
sio.on('reload',(info)=>{
console.log(['sio','reload',info])
document.location.reload()
})
</script>
class App(Web, Module):
def __init__(self, V):
## flask module instance
self.flask = flask
## wrapped application instance
self.app = flask.Flask(self.value)
self.app.config['SECRET_KEY'] = config.SECRET_KEY
## `flask-socketio`
self.sio = SocketIO(self.app)
self.watchfiles()
## inotify reload files after change via `sio(reload)``
def watchfiles(self):
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class Handler(FileSystemEventHandler):
def __init__(self,sio):
super().__init__()
self.sio = sio
def on_modified(self, event):
print([self.on_modified,self,event])
self.sio.emit('reload',[event.src_path,event.event_type,event.is_directory])
self.observer = Observer()
self.observer.schedule(Handler(self.sio),path='static',recursive=True)
self.observer.schedule(Handler(self.sio),path='templates',recursive=True)
self.observer.start()