crypto/tls: interoperability problems between go tls server and microsoft/outlook.com tls (smtp starttls) client #70232
Comments
|
cc @golang/security |
seankhliao
added
the
NeedsInvestigation
|
Possibly a case of https://tldr.fail. |
|
At that stage in the handshake tldr.fail is unlikely. We did have an issue I can’t find at the moment about the size of our tickets with MS stacks, but it’s mostly client certificates that make those grow. |
|
@mjl- Thank you for your report. We were also about to report this at #61721 but I found this issue. So I decided to redirect our report here. We (@stupoid and I) are investigating TLS 1.3 handshake issue with connections from Outlook (Exchange Online). Let us share our findings. In short: Microsoft's TLS 1.3 implementation seems to terminate a TLS connection during the handshake depending on timing when it receives If you are experiencing similar issues with TLS 1.3 handshakes and find that setting Below is the details of our findings. TLS 1.3 Handshake Interoperability IssueOur server Setup# go version
go version go1.22.7 linux/amd64ProblemAs @stupoid describes at #61721 (comment), we were encountering EOF errors during TLS 1.3 handshake from connections from ObservationWe compared how TLS 1.3 handshake went with Go 1.22.7, against an SMTP client and Exchange Online (outlook.com). The following is a dump of a successful handshake with SMTP client ( The following is a dump of a failed handshake with Exchange Online. The handshake didn't finish because the client sent The key difference is the timing of
On a side note, then we tested this with Postfix + OpenSSL ( To verify an assumption that AnalysisWhile I'm not an expert in TLS implementation, I reviewed the spec and found the following: https://datatracker.ietf.org/doc/html/rfc8446#section-4.6.1 says:
and
I think Go's TLS stack follows the second case because the server doesn't request client authentication. On the other hands, Microsoft's TLS stack might expect to receive the server's To verify this hypothesis, I made a small modification to the Go's handshake code to flush the buffer first before sending Here is the patch I tested with: --- src/crypto/tls/handshake_server_tls13.go.orig 2024-11-07 04:28:50.967023405 +0000
+++ src/crypto/tls/handshake_server_tls13.go 2024-11-07 05:02:21.053073557 +0000
@@ -75,9 +75,17 @@
if _, err := c.flush(); err != nil {
return err
}
+
if err := hs.readClientCertificate(); err != nil {
return err
}
+
+ if !hs.requestClientCert() {
+ if err := hs.sendSessionTickets(); err != nil {
+ return err
+ }
+ }
+
if err := hs.readClientFinished(); err != nil {
return err
}
@@ -777,11 +785,11 @@
// If we did not request client certificates, at this point we can
// precompute the client finished and roll the transcript forward to send
// session tickets in our first flight.
- if !hs.requestClientCert() {
- if err := hs.sendSessionTickets(); err != nil {
- return err
- }
- }
+ //if !hs.requestClientCert() {
+ // if err := hs.sendSessionTickets(); err != nil {
+ // return err
+ // }
+ //}
return nil
}It seemed to work. Questions...@FiloSottile, as the author of this code almost 6 years ago, what do you think about this issue? Given these findings, should Go adjust its handshake behavior, or should Microsoft update their TLS 1.3 implementation for better interoperability? |
Just to add to this for anyone looking to sidestep this issue. We encountered really similar issues and also tried what you mentioned by changing Dump of interaction with Exchange Online (outlook.com) with ClientAuth set to Dump of interaction with Exchange Online (outlook.com) with ClientAuth set to |
Go version
go1.23.2 linux/amd64
Output of
go envin your module/workspace:What did you do?
Deploy mox, a mail server, and successfully get incoming email message deliveries from microsoft (outlook.com, both office365 and personal/free accounts) to mox over SMTP with STARTTLS (crypto/tls server).
What did you see happen?
On October 24 I started receiving "TLS reporting" errors with "validation failure" error in the "sts" (MTA-STS) section. Up to and including October 23 I received TLS reports with only successful delivery attempts. I investigated, but couldn't find anything wrong. Yesterday I learned message deliveries from microsoft (outlook.com servers) to mox were failing. The TLS reporting error message wasn't precise/clear, but there's a good chance it was about these failing deliveries attempts.
The symptoms: I would see an incoming smtp connection, the "starttls" command, and an abrupt close of the connection by remote. Debugging revealed the connection was closed by remote after reading the server-side response the the TLS client hello message, without the remote writing anything in response (EOF while trying to read the first bytes looking for the "client finished" message). During more debugging, I noticed the Go TLS server code sends a session ticket message as part of its response to the client hello message. Setting
tls.Config.SessionTicketsDisabled = trueprevents the new session ticket from being sent, and makes the Microsoft SMTP STARTTLS command, and delivery of messages, succeed.At https://datatracker.ietf.org/doc/html/rfc8446#section-4.6.1 I noticed:
One theory: The Go TLS server is sending the NewSessionTicket message too soon, and Microsoft changed their implementation to be more strict about when it allows certain messages.
This isn't specific to mox. Maddy, another mail server written in Go is also seeing TLS interoperability issues with Microsoft/outlook.com. More details:
https://github.com/mjl-/mox/issues/237
foxcpp/maddy#730
What did you expect to see?
The Go TLS session ticket may come too early for some other TLS clients. I did not try changing the crypto/tls code to only send a new session ticket message after having read the client finished message. May be worth trying, to see if that will result in a successful TLS session or sees the same abrupt connection close.
The text was updated successfully, but these errors were encountered: