Gzip vs Brotli: Compression methods compared

Due to the increasing volume of data transmitted over networks such as the Internet, compression methods play an ever more important role. The aim of any compression is to reduce file sizes, which has beneficial effects on file transfer and archiving. Gzip and Brotli are two compression methods that pursue the same goal but operate differently in detail. In the following article we explain the basic principles of both systems and how they can be used optimally.

How does data compression work?

In data compression, the amount of digital data is reduced or condensed. To achieve this, appropriate algorithms remove redundant information from the original files. However, to be able to restore the data completely, the information is converted into a representation that is significantly shorter than the original version.

Both Brotli and Gzip use the dictionary method for this, in which certain keywords are replaced by tokens. The following example text illustrates the principle of this type of compression.

An original text such as

display: Image1; display: Image2

can be compressed to the text

F1: Image1; F1: Image2

The instruction display is associated with the token F1. This token is stored in the dictionary and substitutes display throughout the source text.

The advantage is that the character string has been significantly shortened, and therefore fewer resources are required for archiving or transferring the file.

To restore the original dataset, the file must be decompressed. For this the client requires the corresponding algorithm ("the dictionary") to replace the tokens with the original character strings.

The importance of data compression

Lossless compression can reduce file size without affecting data quality. Both Brotli and Gzip are methods that compress files without loss.

In practice, compression algorithms are of particular importance for the applications of

• data archiving and
• data transmission

.

Data archiving

When it comes to storing large datasets, compression can achieve significant resource savings. Compressed files require less storage space on hardware, allowing it to be used more efficiently and cost-effectively.

Compression algorithms such as Gzip or Brotli are particularly useful for the creation and storage of backups.

Data transmission

Compressing files can achieve a higher data throughput during transmission.

Before data can be exchanged between a web server and an (FTP-) client or web browser, both systems must agree on the compression method used.

When visiting a website this happens as follows. The browser tells the server which methods it recognises. The server replies with the best available method and sends the compressed data. By decompressing, the browser makes them usable for the website visitor.

For the user, the benefit of compression is noticeable in significantly better performance and shorter loading times of the website. Combined with Lazy-Loading and a fast server, webmasters can make visiting their website considerably more pleasant for internet users through data compression.

How Gzip works

Gzip is one of the oldest compression programmes. It was developed in 1992 by Jean-Loup Gailly. Gailly's aim was to replace the compress programme used under Unix. Today Gzip is available under the GPL licence open-source and free for all common operating systems. It uses the open Deflate algorithm and is therefore not subject to any patent rights. Therefore it can be offered as freeware and is available to users free of charge.

Gzip works using the dictionary method. To do this, the program searches for identical parts in a file and joins them using a corresponding dictionary entry. Within a window size of 32 KiB the program looks for an identical sequence of bytes. If a sequence does not repeat within the previous 32 KiB, it is excluded from the compression. It therefore remains uncompressed.

Gzip creates dictionary entries separately for each file, which is somewhat more time-consuming than Brotli's approach. Nevertheless, Gzip is very effective and can achieve a high level of compression depending on the file.

The program creates the compressed file in .gz or .z format.

How Brotli works

The name Brotli is somewhat unusual for software and comes from a Swiss pastry. The system was developed by Zoltán Szabadka and Jyrki Alakuijala. Both were Google employees at the time of its development.

Brotli also works with the dictionary method, but the principle differs from Gzip in the details.

For file compression, Brotli creates hardly any new entries. Instead, it uses a 120 KiB dictionary with over 13,000 predefined strings. These are the most common character sequences that occur in text or HTML documents. Thus, all sequences already known to Brotli can be compressed much faster. Only for unknown sequences do separate entries need to be created. This results in significantly improved compression, especially for small files.

Compared with the Deflate algorithm used by Gzip, Brotli can achieve up to a 20% improved compression rate.

Use cases

In principle, both systems are supported by the most common web servers and web browsers. For data transfer purposes both are therefore equally suitable, also because they can deliver both static and dynamic HTML pages.

In most cases, it will be Brotli, however, that achieves a higher level of compression, making it the better choice for many users.

Our article is based on our own experience and research as well as information from external sources.

Sources & further reading on the topic:
https://github.com/google/brotli (Brotli on github.com)
https://www.gnu.org/software/gzip/ (official Gzip website)