1

Hoping to get a comment from the GAE python team on this one.

Has something changed between 1.6.3, 1.6.4 with regards to the way the dev_appserver datastore is flushed to disk on app exit?

I'm using django-nonrel, and on 1.6.3, and before, I used to be able to load up a python shell: python manage.py shell (manage.py is provided by django-nonrel)

I could then import my models and acceess the datastore, modify entities, and quit.

On 1.6.4, I'm able to do that, but when I quit, changes are not saved to the datastore. When I run django-nonrel as a WSGI app, it saves properly, and I see a message on exit ("Applying all pending transactions and saving the datastore").

4
  • Something did change. I'll ask around. Apr 8, 2012 at 4:07
  • Thanks Guido, I hacked in a fix for django-nonrel. I suspect any other python users who want to get at the datastore via the python shell will have a similar issue.
    – dragonx
    Apr 8, 2012 at 4:20
  • See Alfred Fuller's response to stackoverflow.com/questions/10047315/… for an explanation and source links. Apr 8, 2012 at 14:40
  • @Guido What about handling exit() from the shell? That doesn't send a TERM or KeyInterrupt signal. Is using atexit, like in my answer below correct?
    – dragonx
    Apr 8, 2012 at 23:24

3 Answers 3

3

Thanks to dragonx for his solution and info. I run my devserver from eclipse, and I was amazed to see my data not beeing saved after upgrading to 1.6.4 I added a flush to the database after every web request, to do that I implemented a base class for all requests and override dispatch:

developmentServer = False

if os.environ.get('SERVER_SOFTWARE','').startswith('Development'):
    developmentServer = True

class BaseRequestHandler(webapp2.RequestHandler):
    def dispatch(self):
        retValue = super(BaseRequestHandler, self).dispatch()
        if developmentServer:
                from google.appengine.tools import dev_appserver 
                dev_appserver.TearDownStubs()

        return retValue

informing about a change in behavior like that in the release notes, would have saved me two days of searching what went wrong in my upgrade.

3
  • This code above doesn't work for me on Windows. I've tried launching the application on the development server (1) using Eclipse and (2) using the Google App Engine Launcher. Either way, i.e. (1) terminating the console in Eclipse and (2) stopping the application from the launcher, transactions are not saved to the development datastore. Anyone having similar issues on Windows? Anyone who could offer some thoughts on this?
    – cv12
    Apr 18, 2012 at 5:08
  • I changed the solution to work in eclipse, save the database every time like before google update Apr 22, 2012 at 6:14
  • What I did finally was to create a button on my app that sends the flush when I press it Apr 22, 2012 at 9:17
2

It looks like there have been some changes. I've been able to hack around the problem with the following:

from google.appengine.tools import dev_appserver
import atexit
atexit.register(dev_appserver.TearDownStubs)

This ensures the datastore is flushed on exit.

2
  • If you're running "python manage.py shell" or "python manage.py createsuperuser". These cases weren't working before. If you run "python manage.py runserver", you'll get the flush twice, once from the SDK handler, and one from the extra handler I added. I'm not really sure how to distinguish the python shell exit() (missed by SDK) from SIGTERM/KeyInterrupt (caught by SDK)
    – dragonx
    Apr 9, 2012 at 0:45
  • thanks, I don't know what I did wrong in my post there, I think I just copy and pasted wrong, but the code in my source tree is what you had. I edited it to fix it.
    – dragonx
    Apr 11, 2012 at 5:21
0

Before 1.6.4, we saved the datastore after every write. This method does not work when simulating the transactional model found in the High Replication Datastore (you would lose the last couple writes). It is also horribly inefficient. We changed it so the datastore dev stub flushs all writes and saves it's state on shut down.

Following the code:

  1. https://bitbucket.org/wkornewald/djangoappengine/src/60c2b3339a9f/management/commands/runserver.py#cl-154
  2. http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/tools/dev_appserver_main.py#683

It looks like manage.py should work if the server is shut down cleanly (with a TERM signal or KeyInterrupt).

2
  • That's exactly what I'm seeing. Shutdown with a TERM signal or KeyInterrupt flushes properly. Exiting using exit() in a python shell does not. See my above post. The shell catches KeyInterrupt, so you can't exit that way when using the python shell.
    – dragonx
    Apr 8, 2012 at 14:12
  • 1
    Alfred, can you elaborate on what should happen on Windows? 1.6.4 does not flush neither on py_dev stopping debugging, nor on stopping the Application in the launcher Apr 18, 2012 at 10:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.