Serving compressed JSON on S3 by Paulo Fierro

Last week we shipped Brainfeed, an iPad app we've been working on for a client that presents educational videos for kids. We also developed a backend system for the curators to use to add and tag videos for the app.

This backend is built in Ruby with Padrino (on top of Sinatra) and runs on Heroku. Due to the nature of the app it only needs a single dyno and pushes content changes over to a bucket on Amazon S3 which is consumed by the iPad app. This means that the app can continue to work independently from the backend and any scaling issues can be handled on the S3 side of things, which has the added bonus of keeping the overall cost down.

Previously the app was consuming a JSON file provided by the backend via an API. When you're serving JSON (or any data) you really want to serve it compressed to save on bandwidth and overall load time.

In Apache you can add a line to your .htaccess file that indicates this like so:

AddOutputFilterByType DEFLATE application/json

This gzips JSON content on the fly automatically. This is normally already set in Padrino/Sinatra apps using Rack::Deflater.

However, when I started pushing the JSON over to S3 I needed a way to tell S3 to serve the content in a compressed fashion. Using the AWS SDK for Ruby you can do this like so:

Before the gzip compression we were serving about 960KB. Now that's down to 240KB which is much more manageable. Due to the nature of the app (watching online videos) we know the users are going to be on wi-fi so its not that big a deal, but every little bit helps.

For some reason this took way too long to figure out, so here it is for next time.