The Simple Mail Transfer Protocol (SMTP) is a well-established Internet protocol used by mail servers to send and receive email messages. Since its introduction in 1982, SMTP has remained a plaintext protocol, making it vulnerable to interception or manipulation by malicious actors.
To address this critical weakness, STARTTLS was introduced in 1999, adding support for encryption within SMTP sessions. However, STARTTLS has known limitations and is susceptible to attacks like downgrade attacks, which can strip away the encryption it was meant to provide. To mitigate these weaknesses, the Mail Transfer Agent - Strict Transport Security (MTA-STS) protocol was developed in 2018. MTA-STS is designed to enforce the use of encryption and ensure the authenticity of receiving mail servers, effectively closing the weaknesses of both SMTP and STARTTLS.
Unlike other email security protocols such as Sender Policy Framework (SPF), DomainKeys Identified Mail (DKIM), and Domain-based Message Authentication, Reporting & Conformance (DMARC), implementing MTA-STS involves multiple steps and requires careful testing. As a relatively new standard, it is also less widely understood. In this article, we walk through the process of implementing MTA-STS, share some best practices, and highlight common mistakes to avoid.
The table below summarizes the main concepts related to the MTA-STS protocol that we will cover in this article.
|
Concept |
Description |
|
What is MTA-STS? |
MTA-STS is an email security protocol that was designed to enforce the use of encryption during transit and ensure the authenticity of receiving mail servers, effectively compensating for the weaknesses of both SMTP and STARTTLS. |
|
Anatomy of an MTA-STS DNS record |
Like SPF, DKIM, and DMARC, MTA-STS can be implemented as a DNS TXT record by the domain owner. This DNS record points to the HTTPS endpoint that contains the MTA-STS policy and also signals the sending mail servers when the domain’s MTA-STS policy is updated. |
|
Best practices for the MTA-STS implementation |
When implementing MTA-STS, it is important to follow best practices, such as rotating the “id” tag when the policy changes or testing the policy and using tools to validate the correctness of your implementation before switching to enforce mode. |
|
Common pitfalls and how to avoid them |
Common mistakes like misconfigured MTA-STS policy, expired TLS certificates, and enforcing MTA-STS without proper testing can greatly hinder your MTA-STS implementation. |
|
Monitoring with TLS-RPT |
TLS-RPT is a reporting mechanism that works alongside MTA-STS to help domain owners monitor email delivery issues related to TLS encryption. |
MTA-STS is an email security protocol defined in RFC 8461 that is designed to address the security issues in both SMTP and STARTTLS. Its primary goal is to enforce encrypted email exchange between mail servers while also ensuring the authenticity of the receiving server.
Before a transmitting mail server sends an email to a receiving mail server, it first initiates an unencrypted handshake by sending an EHLO message. The receiving mail server will reply with an EHLO response containing 250 STARTTLS (if supported), telling the sending mail server to upgrade the connection and only send encrypted messages from now on. Because the EHLO message and response are exchanged in plaintext, a man-in-the-middle (MITM) attacker could intercept the EHLO response and drop the 250 STARTTLS part. The sending mail server will interpret the lack of 250 STARTTLS as the inability of the receiving mail server to support Transport Layer Security (TLS) for encrypted communication. As a result, it will continue to communicate with the receiving mail server in plaintext. This attack is known as a STARTTLS downgrade attack, and it allows the threat actor to read, manipulate, or drop plaintext emails that are exchanged between the sending and receiving mail servers.
To solve this problem, the MTA-STS email security protocol was introduced in 2018. By hosting an MTA-STS policy (in enforce mode) on a web server and publishing a DNS TXT record, the receiving mail server signals the sending mail server that it supports TLS and only accepts encrypted emails. Because the MTA-STS policy is hosted on a web server with a valid TLS certificate, unlike the initial unencrypted EHLO handshake, a threat actor cannot tamper with or downgrade this HTTPS connection the same way they would do with STARTTLS in downgrade attacks. This ensures that all email messages exchanged between the mail servers are encrypted and cannot be intercepted by MITM attackers, if any. Furthermore, the HTTPS connection provides the cryptographic assurance of authenticity, ensuring that the MTA-STS policy is retrieved directly from the legitimate domain owner’s web server, thereby preventing interception or tampering by MITM attackers.
Before an organization can set a DNS record for MTA-STS, it first needs to create an MTA-STS policy using the following format:
|
Tag |
Description |
Example |
|
version |
MTA-STS policy version |
STS1v1 |
|
mode |
The MTA-STS policy version can be set to none (no effect), testing (accepts both encrypted and unencrypted emails), or enforce (only accepts encrypted emails). |
testing |
|
mx |
A list of domains’ mail servers that support encrypted communication. More than one can be specified, or a wildcard (*) can be used. |
mail-server1.example.com |
|
max-age |
The duration in seconds for which this MTA-STS policy should be cached by sending mail servers. |
50000 |
This MTA-STS policy should be placed in a file named mta-sts.txt in a special directory named .well-known (note the leading dot). This directory and file are hosted in a special subdomain, mta-sts, of the receiving domain. For example, if the sending mail server would like to send an email to example.com domain, it will first try to retrieve the MTA-STS policy from https://mta-sts.example.com/.well-known/mta-sts.txt. Note that the MTA-STS protocol requires that the web server have a valid TLS certificate issued by a trusted certificate authority (CA) and that it supports TLS v1.2 or higher.
After creating and hosting the MTA-STS policy, your organization should create a DNS TXT record named _mta-sts. This DNS record must contain a v tag for MTA-STS version and the id tag, which represents the last date/time the MTA-STS policy was changed:
|
v=STSv1; id=20250825231200Z |
Finally, your organization needs to make sure that all the listed mail servers on the MTA-STS policy are configured to handle encrypted communications, e.g., they have valid TLS certificates. This is rarely an issue, as most modern mail servers already provide built-in support for this. You can verify the correctness of your configuration with different online tools.
Once the steps above are completed successfully, your organization will be able to receive encrypted email from other mail servers. Before sending an email, each sending mail server will check for the existence of the _mta-sts DNS record for the receiving domain, retrieve and verify the content of the MTA-STS policy hosted on mta-sts.<some_receiving_domain_name>, and send encrypted emails to any of the receiving mail servers. Note that if the receiving mail server replies to the EHLO message without 250 STARTTLS during the handshake, the sending mail server will understand that something is wrong and refuse to send the email in plaintext, thwarting any STARTTLS downgrade attempt.
MTA-STS workflow performed by the sending mail server
Compared to SPF, DKIM, and DMARC, implementing MTA-STS demands more effort and thorough testing. It is essential to follow best practices during deployment to maintain uninterrupted email deliverability.
MTA-STS policy allows domain owners to choose between none, testing, and enforce modes. Before switching to enforce mode, it is recommended to set the policy to testing mode and use TLS-RPT reports (covered in another section) to troubleshoot any email delivery issues for a certain period of time. You can also use free online tools to check whether your MTA-STS implementation is working properly. After ensuring that all emails are received without issues, the policy can be switched to enforce mode to fully benefit from MTA-STS.
Every time the MTA-STS policy is modified, the id tag of the DNS TXT record for MTA-STS should be modified as well to reflect the current date/time of the change. This is important since, if this is not changed accordingly, sending mail servers will keep using the cached/old version of the MTA-STS policy to send encrypted emails, which might lead to email delivery issues. By changing the id tag, you signal the sending mail server of a policy change, which will then refetch and cache the new MTA-STS policy.
Unlike SPF, DKIM, and DMARC, setting up MTA-STS involves more than simply creating a DNS TXT record. When hosting the MTA-STS policy on your web server, certain things must be taken into consideration:
Defined in RFC 8460 in 2018, SMTP TLS Reporting (TLS-RPT) provides functionality similar to DMARC reporting. These JSON-formatted reports are typically sent daily by sending mail servers and report any issues related to TLS negotiation or MTA-STS policy that may have impacted the secure delivery of encrypted emails to your mail servers.
More specifically, a TLS-RPT report contains information such as the reporting source, the affected domain, the affected receiving mail server, the report period, failure type, failure reason, a count of affected messages, and the result (i.e., email delivered or not). These reports are especially useful when MTA-STS is first set up in testing mode, as it helps debug delivery issues before switching to enforce mode.
To receive TLS-RPT reports, your organization must set up an additional DNS TXT record named _smtp._tls. This record consists of only two fields: the v field for the version of TLS-RPT and the rua field that contains the email addresses that will receive the TLS-RPT reports. For example, the _smtp_.tls.example.com DNS record for a domain like example.com would look like this:
v=TLSRPTv1; rua=mailto:<some_email_address>@example.com
Once set up, it is recommended that your organization regularly reviews these reports to ensure that no email delivery issues go unnoticed and are handled accordingly.
MTA-STS is an important protocol that strengthens email security by enforcing encryption and ensuring messages are only delivered to trusted receiving mail servers. In this article, we explored what MTA-STS is, how it is implemented, and some best practices and common pitfalls to consider during implementation. Finally, we highlighted the importance of configuring and analyzing TLS-RPT reports for ongoing visibility into mail delivery issues. By properly setting up and maintaining MTA-STS, your organization can significantly reduce the risk of email interception and downgrade attacks by MITM attackers, helping ensure secure and reliable email communication.