What is StartTLS and how does it affect encryption?

Encrypting communication over distributed networks such as the Internet is essential for several reasons - whether it's your own email domain, authentication on websites, or transferring files and documents. An open and unencrypted connection allows any participating server and node to store and analyse this information transmitted in plain text. To ensure security, transmission requires secure methods. Different methods and protocols have been established for this purpose, including Secure Shell, Secure Sockets Layer (SSL), Transport Layer Security (TLS), and its variant, StartTLS. The variety of options often leads to confusion about which abbreviation corresponds to which method and ensures maximum security for users and servers.

What role does StartTLS play in encryption?

StartTLS is not a separate network protocol but rather a defined procedure with universal commands that initiates encryption via TLS and negotiates the conditions for bilateral communication between the involved endpoints. It has some distinctive features that set it apart from other methods. Features of StartTLS include:

• Encryption using various versions of TLS
• Establishing a connection initially in an unencrypted manner
• Compatibility with different standards, including unencrypted data exchange
• Servers and clients jointly define conditions for communication
• Application for websites, external and internal email domains
• User authentication through the negotiated protocol
• Vulnerability to attacks by third parties acting as intermediaries (Man-in-the-Middle Attacks)

The establishment of a connection via StartTLS initially occurs unencrypted in plain text to ensure extensive backward compatibility. During the first request, the client inquires with the server about the support for cryptographic methods and which encryption it offers in SSL or TLS. If both sides use a compatible standard, they then negotiate the protocol and its version via StartTLS to encrypt the authentication and all subsequent information based on this principle.

For more information on the difference between SSL vs TLS, click here.

Criticism of StartTLS as a method

The main issue with StartTLS is that it does not initially use encryption but only implements it in a second stage. Therefore, it is a method for explicit rather than implicit security, known as opportunistic encryption. In contrast, implicit TLS uses full encryption from the first individual request by encrypting packets with a public key or certificates that can only be decrypted back into plain text by an authorised counterpart.

Given the background of StartTLS, its approach is logical - its origin dates back to the turn of the millennium. At that time, communication in public networks, with few exceptions in security-critical areas such as the transmission of banking data, was not encrypted. To increase the general dissemination and security of communication, an interoperable standard compatible with insecure clients like StartTLS proves advantageous. However, it inevitably represents a temporary rather than a permanent solution.

The situation has significantly changed 20 years later - it is now generally not advisable to transmit data without secure encryption, and common browsers and servers support suitable methods with rare exceptions. For this reason, requests transmitted in plain text as with StartTLS have become obsolete in multiple respects. Given today's threat landscape and the highly specialised attacks, it is generally safer in most cases not to transmit data rather than to send it without adequate protection - especially during any authentication involving a combination of username and password.

What are the main use cases of StartTLS?

Although StartTLS can also be used for websites, it is primarily used to encrypt communication via email. Many email clients such as Mozilla Outlook or Thunderbird still support the procedure to determine the options for establishing a secure connection to a server. The initial query is made as a StartTLS command over the standard connection before authentication and message transmission occur on a specific channel (port) that the server communicates to the client.

One reason for this specialization is that some mail servers did not offer any encryption for a long time or worked with competing, incompatible standards. For this reason, StartTLS proved to be a useful addition during this period, capable of increasing security to the highest possible level. However, since 2010, it is generally advisable to avoid using unencrypted connections, and reputable, security-focused email providers universally support full encryption via TLS. For these reasons, the use of StartTLS should generally be avoided in favour of TLS - both as a client and on VPS hosting, Cloud and Dedicated Servers.

Photo: Tumisu from Pixabay