Gzip vs Brotli: A Comparison of Compression Methods

Due to the increasing amount of data being transferred over networks like the Internet, compression methods are playing an increasingly important role. The goal of compression is to reduce file sizes, which has positive effects on file transfer and archiving. Gzip and Brotli are two compression methods that aim to achieve the same goal but work differently in detail. In the following article, we will show the basic principles of both systems and how they can be optimally used.

How Does Data Compression Work?

Data compression involves reducing or condensing the amount of digital data. To achieve this, specific algorithms remove redundant information from the original files. However, to fully restore the data, the information is transformed into a representation that is significantly shorter than the original version.

Both Brotli and Gzip use the dictionary method, where specific keywords are replaced by tokens. The following example text illustrates the principle of this type of compression.

An original text like

display: Image1; display: Image2

can be compressed to

F1: Image1; F1: Image2

The instruction display is associated with the token F1. This token is stored in the dictionary and substitutes display throughout the entire source text.

The advantage is that the string has been significantly shortened, requiring fewer resources for archiving or transferring the file.

To restore the original dataset, the file needs to be decompressed. The client requires the corresponding algorithm ("the dictionary") to replace the tokens with the original strings again.

The Importance of Data Compression

Lossless compression can reduce file size without compromising data quality. Both Brotli and Gzip are methods used to compress files without loss.

In practice, compression algorithms are particularly important for the applications of

• Data archiving and
• Data transmission

When it comes to storing large datasets, compression can achieve significant resource savings. Compressed files require less storage space on hardware, making it more efficient and cost-effective.

Especially for creating and storing backups, compression algorithms like Gzip or Brotli are very useful.

Data Transmission

Compressing files can increase data throughput during transmission.

Prior to exchanging data between a web server and an (FTP) client or web browser, both systems must agree on the compression method used.

When visiting a website, this process occurs as follows. The browser informs the server of the methods it knows. The server responds with the best possible method and sends the compressed data. The browser then decompresses it for the website visitor to use.

For the user, the benefit of compression is noticeable in significantly improved performance and shorter website loading times. Combined with Lazy-Loading and a fast server, webmasters can make the visit to their website much more enjoyable for internet users through data compression.

How Gzip Works

Gzip is one of the oldest compression programs. It was developed in 1992 by Jean-Loup Gailly. Gailly's aim was to replace the compress program used under Unix. Today, Gzip is open source and freely available for all common operating systems. It uses the free Deflate algorithm and is not bound by any patent rights. Therefore, it can be offered as freeware and is available to users free of charge.

Gzip operates using the dictionary method. The program searches for identical parts in a file and combines them using a corresponding dictionary entry. In a window size of 32 KiB, the program looks for an identical sequence of bytes. If a sequence does not repeat within the previous 32 KiB, it is excluded from compression and remains uncompressed.

Gzip creates dictionary entries separately for each file, which is slightly more time-consuming than Brotli's method. Nevertheless, Gzip works very effectively and can achieve a high level of compression depending on the file.

The program creates the compressed file in the format .gz or .z.

How Brotli Works

The name Brotli is somewhat unusual for software and comes from a Swiss pastry. The system was developed by Zoltán Szabadka and Jyrki Alakuijala, both of whom were Google employees at the time of development.

Like Gzip, Brotli also operates using the dictionary method, however, the principle differs from Gzip in detail.

For compressing files, Brotli hardly creates new entries. Instead, it uses a 120 KiB large dictionary with over 13,000 predefined expressions. These are the most common character strings found in text or HTML documents. This allows all data that Brotli already knows to be compressed much faster. Separate entries only need to be created for unknown expressions. This results in significantly improved compression, especially for small files.

Compared to the Deflate algorithm used by Gzip, Brotli can achieve an improved compression rate of up to 20%.

Applications

Both systems are generally supported by the most common web servers and web browsers. Therefore, both are equally suitable for data transfer, as they can deliver both static and dynamic HTML pages.

In most cases, Brotli will achieve a higher level of compression, making it the better choice for many users.

Our article is based on our own experiences and research as well as information from external sources.

Sources & further links on the topic:
https://github.com/google/brotli (Brotli on github.com)
https://www.gnu.org/software/gzip/ (official Gzip website)

Image credit:
Gerd Altmann on Pixabay