1

I am working on a website that I want to host on App Engine. My App Engine scripts are written in Python. Now let's say you could register on my website and have a user profile. Now, the user Profile is kind of extensive and has more than 50 different ndb properties (just for the sake of an example).

If the user wants to edit his records (which he can to some extend) he may do so through my website send a request to the app engine backend.

The way the profile is section, often about 5 to 10 properties fall into a small subsection or container of the page. On the server side I would have multiple small scripts to handle editing small sections of the whole Profile. For example one script would update "adress information", another one would update "interests" and an "about me" text. This way I end up with 5 scripts or so. The advantage is that each script is easy to maintain and only does one specific thing. But I don't know if something like this is smart performance wise. Because if I maintain this habbit for the rest of the page I'll probably end up with about 100 or more different .py scripts and a very big app.yaml and I have no idea how efficiently they are cached on the google servers.

So tl;dr:

Are many small backend scripts to perform small tasks on my App Engine backend a good idea or should I use few scripts which can handle a whole array of different tasks?

3 Answers 3

4

The are two important considerations here.

  1. The number of roundtrip calls from the client to the server.

One call to update a user profile will execute much faster than 5 calls to update different parts of user profile as you save on roundtrip time between the client and the server and between the server and the datastore.

  1. Write costs.

If you update 5 properties in a user profile and save it, and then update 5 other properties and save it, etc., your writing costs will be much higher because every update incurs writing costs, including updates on all indexed properties - even those you did not change.

Instead of creating a huge user profile with 50 properties, it may be better to keep properties that rarely change (name, gender, date of birth, etc.) in one entity, and separate other properties into a different entity or entities. This way you can reduce your writing costs, but also reduce the payload (no need to move all 50 properties back and forth unless they are needed), and simplify your application logic (i.e. if a user only updates an address, there is no need to update the entire user profile).

3

A single big script would have to be loaded every time an instance for your app starts, possibly hurting the instance start time, the response time of every request starting an instance and the memory footprint of the instance. But it can handle any request immediately, no additional code needs to be loaded.

Multiple smaller scripts can be lazy-loaded, on demand, after your app is started, offering advantages maybe appealing to some apps:

  • the main app/module script can be kept small, which keeps the instance startup time short
  • the app's memory footprint can be kept smaller, handler code in lazy-loaded files is not loaded until there are requests for such handlers - interesting for rarely used handlers
  • the extra delay in response time for a request which requires loading the handler code is smaller as only one smaller script needs to be loaded.

Of course, the disadvantage is that some requests will have longer than usual latencies due to loading of the handler scripts: in the worst case the number of affected requests is the number of scripts per every instance lifetime.

Updating a user profile is not something done very often, I'd consider it a rarely used piece of functionality, thus placing its handlers in a separate file looks appealing. Splitting it into one handler per file - I find that maybe a bit extreme. It's really is up to you, you know better your app and your style.

From the GAE (caching) infra perspective - the file quota is 10000 files, I wouldn't worry too much with just ~100 files.

2
  • Following your answer: Right now my app doesn't have a main app/module. I have a client app that interacts with the backend solely by doing ajax requests basically. When a user logs in there is a POST request to the login.py module that returns a Token. When a user signs up there is a POST Request to my signup.py. And so on and so forth. They are all seperate .py files which have seperate names in my app.yaml file. So I assume when one instances can't handle the incoming requests to all my different scripts a new one will start up and eventually cache all my scripts too?
    – Escapado
    Oct 31, 2015 at 22:33
  • 1
    Yes any instance can serve any of the handlers in your app.yaml. Other than startup time it really doesn't make a lot of difference between one single handler and many handlers. I use a single handler because of the URL routing model I use is traversal based using pyramid. If your using something like webapp2 it will tend towards having many small handlers registered with app.yaml. Both work, it one more appropriate than the other - depends . Nov 1, 2015 at 5:59
0

Adding to Dan Cornilescu’s answer, writing/saving an instance to the database re-writes to the whole instance (i.e. all its attributes) to the database. If you’re gonna use put() multiple times, you’re gonna re-write the who instance multiple times. Which, aside from being a heavy task to perform, will cost you more money.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.