Possible low hanging optimization fruit

[foxfirefey]

So, I've been doing some heavy web page optimization at my day job and on a whim I ran an analyzer on some DW pages and found what I think could be a very easy-to-make-happen optimization: we're not gzipping the JS/CSS static files when we serve them.

Examples:

* The new entry page: 221.2KiB (70% reduction) -- quite the savings when the entire bundle is 349.9 KB
* My reading page: 149.3KiB (68% reduction) -- when the total page is 541.5 K, so decent

Would this be as easy to set up as I think it would, or are there other reasons not to do it?

Comments

[mark]

I am pretty sure it can't, but I'd be excited to learn otherwise!

[alierak]

Commence excitement!

[foxfirefey]

Eeee!

[mark]

varnishd (varnish-3.0.2 revision cbf1284)

EXCITEMENT. Added to the config:

    # gzip outgoing HTML/CSS/JS.
    if (req.url ~ "\.(html|css|js)(\?v=\d+)?$" &&
            beresp.http.content-type ~ "text") {
        set beresp.do_gzip = true;
    }

So far, it looks good. I see static content is responding to gzip requests, and varnishd's CPU is not crazy higher. It might be up a bit, but it still gets lost in the noise of the Perlbals.

I'm going to be offline for the next ~24 hours. If something is wrong, you can revert the above section from lb02's /etc/varnish/default.vcl and run /root/bin/reload-varnish-config on that machine. (I'm quite sure you know all that, but meh.)

Edit: This doesn't really compress most of our web output. Just the static files. It'd be nice to figure out if we can compress our HTML output, although pattern-matching all of that might be interesting.

We should probably make it a compression-opt-out instead of choosing what to compress. That way we just have to match the media URLs and also PNG, GIF, JPG, etc.

[alierak]

Yay, fun! Looks like js was content-type "application/x-javascript" so I changed that regexp to "text|javascript".

[alierak]

So, apparently we already do compress most of the dynamic html on the site.

root@sb-web03:~# cat /etc/apache2/mods-enabled/deflate.conf 
<IfModule mod_deflate.c>
          AddOutputFilterByType DEFLATE text/html text/plain text/xml
</IfModule>

But, it gets weirder. Some of the webservers have this kind of stuff instead:

root@sb-web01:~# cat /etc/apache2/mods-enabled/deflate.conf
&;lt;IfModule mod_deflate.c>
          # these are known to be safe with MSIE 6
          AddOutputFilterByType DEFLATE text/html text/plain text/xml

          # everything else may cause problems with MSIE 6
          AddOutputFilterByType DEFLATE text/css
          AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript
          AddOutputFilterByType DEFLATE application/rss+xml
</IfModule>

Thanks to Varnish, we've presumably been caching some gzipped stuff all along, but not actively preferring it.