Gzip vs Brotli: Compression algorithms compared

Due to the increasing volume of data transmitted over networks such as the Internet, compression methods play an ever greater role. The aim of any compression is to reduce file sizes, which has positive effects on the transfer and archiving of files. Gzip and Brotli are two compression methods that pursue the same goal but operate differently in detail. In the following article we show the basic principles of both systems and how they can be used optimally.

How does data compression work?

Data compression reduces or condenses the amount of digital data. To achieve this, appropriate algorithms remove redundant information from the original files. To be able to fully restore the data, information is converted into a representation that is significantly shorter than the original version.

Both Brotli and Gzip use the dictionary method for this, in which certain keywords are replaced by tokens. The following example text illustrates the principle of this type of compression.

An original text such as

display: Image1; display: Image2

can be compressed to the text

F1: Image1; F1: Image2

The directive display is assigned to the token F1. This token is stored in the dictionary and substitutes display throughout the source text.

The advantage is that the character string has been significantly shortened, meaning fewer resources are needed for archiving or transmitting the file.

To restore the original dataset, the file must be decompressed. For this the client requires the corresponding algorithm ("the dictionary") to replace the tokens with the original character strings.

The importance of data compression

Lossless compression can reduce file size without affecting data quality. Both Brotli and Gzip are methods that compress files without loss.

In practice, compression algorithms are of particular importance for the applications of

• data archiving and
• data transfer

in particular.

Data archiving

When it comes to storing large datasets, compression can achieve significant resource savings. Compressed files require less storage space on the hardware, allowing it to be used more efficiently and cost-effectively.

Especially for creating and storing backups, compression algorithms such as Gzip or Brotli are of great benefit.

Data transfer

Compressing files can achieve a higher data throughput during transfer.

Before data can be exchanged between a web server and an (FTP) client or web browser, both systems must agree on the compression method to be used.

When visiting a website this happens as follows. The browser tells the server which methods it recognises. The server replies with the best available method and sends the compressed data. By decompressing them the browser makes them usable for the site visitor.

For the user, the advantage of compression is noticeable in significantly better performance and shorter loading times for the website. Combined with lazy loading and a fast server, webmasters can make visiting their website significantly more pleasant for internet users through data compression.

How Gzip works

Gzip is one of the oldest compression programmes. It was developed in 1992 by Jean‑Loup Gailly. Gailly's aim was to replace the compress utility used under Unix. Today Gzip is under the GPL licence open-source and freely available for all common operating systems. It uses the free Deflate algorithm and is therefore not subject to any patent rights. Consequently it can be offered as freeware and is available to users free of charge.

Gzip uses the dictionary method. The programme searches for identical parts within a file and consolidates them using corresponding dictionary entries. Within a window size of 32 KiB the programme looks for an identical sequence of bytes. If a sequence does not repeat within the previous 32 KiB, it is excluded from compression. In other words, it remains uncompressed.

Gzip creates the dictionary entries separately for each file, which is somewhat more time-consuming than Brotli's approach. Nevertheless, Gzip is very effective and can achieve a high degree of compression depending on the file.

The programme produces the compressed file in .gz or .z format.

How Brotli works

The name Brotli is somewhat unusual for software and comes from a Swiss pastry. The system was developed by Zoltán Szabadka and Jyrki Alakuijala. At the time of development both were Google employees.

Brotli also uses the dictionary method, but the principle differs from Gzip in the details.

For file compression Brotli hardly creates new entries. Instead it uses a 120 KiB dictionary with over 13,000 predefined strings. These are the most common character strings found in text or HTML documents. Consequently, all datasets that Brotli already recognises can be compressed much more quickly. Only for unknown strings do separate entries need to be created. This results in significantly improved compression, particularly for small files.

Compared with the Deflate algorithm used by Gzip, Brotli can achieve up to a 20% improved compression rate.

Applications

In principle both systems are supported by the most common web servers and web browsers. For data transfer both are therefore equally suitable, also because they can serve both static and dynamic HTML pages.

In most cases Brotli will achieve a higher degree of compression, which is why it is the better choice for many users.

Our article is based on our own experience and research as well as information from external sources.

Sources & further reading on the topic:
https://github.com/google/brotli (Brotli on github.com)
https://www.gnu.org/software/gzip/ (official Gzip website)