12

I am creating a mobile service based on Pyramid framework. Because it's mobile everything to reduce bandwidth usage is plus. I am considering gzipping all the traffic, even dynamic HTML pages.

What kind of hooks Pyramid framework provides for this? Or is there WSGI middleware for the task? I'd like to do this still on Python level, not Nginx/Apache, so I can better statistics how much gzip brings benefirts.

3 Answers 3

24

First of all I should stress that you should do this on the web server level (nginx or apache). There are several reasons for this:

  1. Performance - If you do this in Python you are using one of your threads that could be handling requests to do cpu-intensive compression. This is way less efficient than allowing your optimized web server to handle it.

  2. Blocking - Most GZip middleware will block your responses, buffering the body so that it can compress the entire response. This is a huge problem if you are attempting to stream any response back to the client because it will get caught in middleware. This is actually a violation of PEP333, the WSGI specification.

With all of this in mind, it might make sense to do it in Python at least for debugging purposes during development.

Since you're already using Pyramid then you have Paste installed. Thus you can simply add the paste.gzipper.GzipMiddleware to your application's pipeline like so:

[filter:gzip]
use = egg:Paste#gzip
compress_level = 6

[pipeline:main]
pipeline =
    gzip
    app

Obviously if you don't want to change the compression level from the default of 6 you can simply add the egg:Paste#gzip to the pipeline in place of configuring the filter and giving it a custom name (gzip).

3
  • 2
    p.s. use RAW DEFLATE if possible (apache's mod_deflate sends GZIP and GZIP only). GZIP is an unnecessary wrapper around RAW DEFLATE. It is always larger and requires an extra checksum calculation. Jul 11, 2011 at 11:43
  • 2019 update request: nowadays, a Pyramid app will have paste.deploy installed, but not all the rest of Paste. Is there another easy way? or should one use a WSGI app server that has gzip feature (waitress doesn’t)? (This is for contexts where you deploy only app code, so can’t use nginx)
    – merwok
    Dec 31, 2019 at 17:24
  • You can still install pip install paste fwiw. Jan 3, 2020 at 5:57
5

You can still get per-request compression statistics with Apache. I created a deflate.log like so:

DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateFilterNote Ratio ratio
LogFormat '%a %v %{outstream}n/%{instream}n (%{ratio}n%%) "%r"' deflate
CustomLog /var/log/httpd/deflate.log deflate

Now I get log entries like:

ip domain -/- (-%) "GET /wp-content/uploads/2010/03/favicon.ico HTTP/1.1"
ip domain 10995/52406 (20%) "GET /2006/07/19/ HTTP/1.0"
ip domain 1873/7891 (23%) "POST /registration/regForm HTTP/1.1"

that I can analyze to my heart's content.

0
1

Here are a couple of options at the WSGI level:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.