40X your RESTful transfer speeds

There is a something fascinating about small (smart) tweaks that bring about disproportionately big changes. This tweak is simple, with drastic results and if used in mobile apps can dramatically help battery-life and bandwidth hog.

REST has been a great boon in general but one of the issues with JSON is that it is verbose and wasteful, especially if you are sending large amounts of tabular data. The same keys seems to repeat often.

One simple tactic which I am surprised to see is not used as often as it should be is to compress the data, using any standard data-compression library, convert it to base64 and send it in response from RESTful endpoints. JSON is naturally a great candidate for this tactic, due to its nature of repeating keys, especially again, for tabular data.

Of course, this tactic becomes exponentially more effective as the amount of data grows.

Test Results:

3KB JSON converts to 800 bytes compressed data, and only makes marginal difference to user experience. But if many concurrent request of this size are being made then it makes a lot of sense.

3MB compresses to 8KB, gets transferred in sub-second time interval and if used in a mobile devices can significantly helps conserve both battery-life and bandwidth!!

Although rare, but if need be, even 300 MB of data can be compressed to as low as under 1MB can be transmitted in just a few seconds, making such large chunks of data transfer possible within the scope of a simple HTTP request, without hitting timeouts.

So don’t be shy, Compress and Transmit!