Developer Edition (DE) not work OpenVPN when using TLS 1.3

[lucpv]

Hi all,
I'm configuring SoftEther VPN Server (Developer Edition - DE) to work with OpenVPN Client 2.6.git using TLS 1.3 handshake protocol, but not work. Same configuration but when used with Stable Edition (SE) works fine. In addition, I have also configured SoftEther VPN Server (DE version) to work successfully with OpenVPN CLient using TLS 1.2 handshake protocol.

To change the configuration between the TLS 1.2 and 1.3 handshake versions, I change the client-side configuration file as follows:
tls-version-min 1.2
tls-version-max 1.2
or
tls-version-min 1.3
tls-version-max 1.3

Thanks everyone for the suggestion.

[VanderShore]

Hi,

What I noticed during testing with the TLS versions is that 1.3 is disabled even if you set the bool Tls_Disable1_3 false.
When I looked through the code I noticed the following line: https://github.com/SoftEtherVPN/SoftEtherVPN/blob/master/src/Mayaqua/Network.c#L5727

SSL_CTX_set_options(ssl_ctx, SSL_OP_NO_TLSv1_3); // For some reason pppd under linux doesn't like it

It originates from the following pull request: #1391

When I comment out the line it works for TLS 1.3.

[domosekai]

This workaround was introduced in #1109 .
@Evengard , can you clarify what issue did you see if enabling TLS 1.3?

[Evengard]

As far as I remember, pppd under linux not compiled with TLS 1.3 support (which was a majority back at the time, maybe still is but not sure) wasn't able to complete the auth, erroring instead. It was kind of long time ago, so I can't remember the exact error, but if my memory serves me well, it detected that the TLS version was too high and refused continuing at all.

[domosekai]

Thank you for the information. It's weird that a client can refuse higher TLS version.
Can you recall the Linux distro you were testing on?

[Evengard]

I'm usually going with Debian distros, this one may be done with WSL2+Debian I think, I'd say around Debian 10 version.

[Evengard]

Found it. https://github.com/ppp-project/ppp/blob/76016e1b948b7d9675b4e0750d1f943d96d9523b/pppd/eap-tls.c#L1164
Here, if it doesn't find the correct version for the EAP auth, it just errors as "unknown version", and TLS 1.3 is indeed under a define here.

[Evengard]

https://github.com/ppp-project/ppp/blob/76016e1b948b7d9675b4e0750d1f943d96d9523b/pppd/eap-tls.c#L1064 here too, TLS 1.3 isn't even an option here.

[Evengard]

https://github.com/ppp-project/ppp/blob/4e895b5d9727fbbbf7c50c6ceedea5139da85f5d/pppd/tls.c#L295
Here it is stated EAP-TLS with 1.3 is still highly experimental (yet), so I guess it's not included in most PPPD packages.

Maybe a better solution would be some kind of flag to override TLS 1.3 only for EAP-TLS.

[VanderShore]

It should be possible to honor the SSL_ACCEPT_SETTINGS by passing it as argument:

For Proto_OpenVPN.c and Proto_PPP.c

c->SslPipe = NewSslPipeEx(true, s->Cedar->ServerX, s->Cedar->ServerK, s->Dh, true, &c->ClientCert, &s->Cedar->SslAcceptSettings);

Then you can disable it as you configure it, similar what is done in StartSSLEx3

[domosekai]

I tested on Win 11 SSTP client which has TLS 1.3 support in EAP-TLS. It seems there is some bug on our side. Not sure about Linux.
We currently require that a final EAP Ack is sent at last from the peer.

SoftEtherVPN/src/Cedar/Proto_PPP.c

Lines 3476 to 3478 in 8362637

	// It probably should be the final ACK on closed SSL pipe
	SyncSslPipe(p->Eap_TlsCtx.SslPipe);
	if (p->Eap_TlsCtx.ClientCert.X != NULL)

But in TLS 1.3 the client can send some data at last, which is not properly handled by us.

Got EAP-TLS size=256
Sent EAP-TLS size=1492 type=13 flag=192
Got EAP-TLS size=1
Data=00
Sent EAP-TLS size=1492 type=13 flag=64
Got EAP-TLS size=1
Data=00
Sent EAP-TLS size=1284 type=13 flag=0
Got EAP-TLS size=1190
SslCertVerifyCallback preverify error: 'unable to get local issuer certificate'
Sent EAP-TLS size=160 type=13 flag=0
Got EAP-TLS size=29  <--- NOT 1
/root/repo/SoftEtherVPN/src/Mayaqua/Network.c: 5996
SyncSslPipe: s->SslInOut error.

In TLS 1.2 the log looks like this:

Got EAP-TLS size=256
Sent EAP-TLS size=1492 type=13 flag=192
Got EAP-TLS size=1
Data=00
Sent EAP-TLS size=1492 type=13 flag=64
Got EAP-TLS size=1
Data=00
Sent EAP-TLS size=1132 type=13 flag=0
Got EAP-TLS size=1200
SslCertVerifyCallback preverify error: 'unable to get local issuer certificate'
Sent EAP-TLS size=53 type=13 flag=0
Got EAP-TLS size=1
Data=00
Accepted.

[Evengard]

I'll try to find some time to fiddle around with that, assign me pls.

[domosekai]

I managed to get Win 11 SSTP working by using an ugly goto. It's not the final solution but at least it shows something positive.
By the way Win 2022 as a EAP-TLS server does not support TLS 1.3.

diff --git a/src/Cedar/Proto_PPP.c b/src/Cedar/Proto_PPP.c
index 96094b70..d792f9b7 100644
--- a/src/Cedar/Proto_PPP.c
+++ b/src/Cedar/Proto_PPP.c
@@ -3480,7 +3480,7 @@ bool PPPProcessEAPTlsResponse(PPP_SESSION *p, PPP_EAP *eap_packet, UINT eapTlsSi
                                IPC *ipc;
                                ETHERIP_ID d;
                                UINT error_code;
-
+LABEL_AUTH:^M
                                /*if (!p->Eap_TlsCtx.SslPipe->IsDisconnected)
                                {
                                        dataSize = FifoSize(p->Eap_TlsCtx.SslPipe->RawOut->RecvFifo);
@@ -3664,6 +3664,12 @@ bool PPPProcessEAPTlsResponse(PPP_SESSION *p, PPP_EAP *eap_packet, UINT eapTlsSi
                        Debug("\n=======RECV EAP-TLS PACKET FIFO END=======\n");*/
                        WriteFifo(p->Eap_TlsCtx.SslPipe->RawIn->SendFifo, dataBuffer, dataSize);
                        SyncSslPipe(p->Eap_TlsCtx.SslPipe);
+^M
+                       if (p->Eap_TlsCtx.ClientCert.X != NULL)^M
+                       {^M
+                               goto LABEL_AUTH;^M
+                       }^M
+^M
                        // Delete the cached buffer after we fed it into the pipe
                        if (p->Eap_TlsCtx.CachedBufferRecv != NULL)
                        {
diff --git a/src/Mayaqua/Network.c b/src/Mayaqua/Network.c
index 7d2ea3db..b96a413e 100644
--- a/src/Mayaqua/Network.c
+++ b/src/Mayaqua/Network.c
@@ -5723,10 +5723,6 @@ SSL_PIPE *NewSslPipeEx2(bool server_mode, X *x, K *k, LIST *chain, DH_CTX *dh, b
        {
                if (server_mode)
                {
-#ifdef SSL_OP_NO_TLSv1_3
-                       SSL_CTX_set_options(ssl_ctx, SSL_OP_NO_TLSv1_3); // For some reason pppd under linux doesn't like it
-#endif
-
                        if (chain == NULL)
                        {
                                AddChainSslCertOnDirectory(ssl_ctx);

[Evengard]

Thanks for the heads up, I'll try to check both Win and PPPD under Linux for a better solution.

[Evengard]

I'm actually having trouble connecting with Win11 built-in SSTP client. Getting weird error "The Local Security Authority cannot be contacted" after EAP-TLS. Thanks Windows for such unclear messages...

Disabling TLS 1.3 actually makes that working again.

After some googling, it seems like this message originates from SCHANNEL, the underlying library handling TLS itself, and that the support for 1.3, while should be present in recent versions in Windows, is kinda finnicky.

Enabling it as specified in https://stackoverflow.com/questions/56072561/how-to-enable-tls-1-3-in-windows-10/59210166#59210166 (for both client and server, actually) didn't help (and probably wouldn't - as it should be already enabled by default on Win11, according to https://learn.microsoft.com/en-us/windows/win32/secauthn/protocols-in-tls-ssl--schannel-ssp-).

I even thought my Windows is borked, but after installing a fresh Win11 into a virtual machine, the result is the same - same error.

I have a feeling we're getting screwed by cypher suites here, trying to negotiate a TLS 1.3 with a blacklisted cypher suite. And the actual 29 bytes response actually led to an error message "tlsv1 alert access denied" from OpenSSL last error - so the 29 bytes response is most likely a symptom of an error anyway.

I may be wrong, if anyone have ideas I'd be happy to hear.

I can rework NewSslPipe to pass a TLS version restriction for EAP-TLS (hardcoding it for now), that way OpenVPN will be unaffected.

[Evengard]

Good news is that pppd seems to handle TLS 1.3 auto downgrade to 1.2 just fine, tested without the workaround with enabled TLS 1.3 against PPPD, and it passed auth without any problems.

[Evengard]

Even more, PPPD actually succeeded authenticating fully in 1.3 flow. Honestly, something is wrong with Windows, but I don't know what.

[Evengard]

For reference, here the PPPD flow with 1.3 against SoftEther

EAP-TLS: Setting max protocol version to 0x304
Initializing SSL BIOs
 -> SSL/TLS Header: TLS 1.0
 -> Handshake: Client Hello
sent [EAP Response id=0x2 TLS --- ...]
rcvd [EAP Request id=0x3 TLS LM- ...]
sent [EAP Response id=0x3 TLS Ack]
rcvd [EAP Request id=0x4 TLS -M- ...]
sent [EAP Response id=0x4 TLS Ack]
rcvd [EAP Request id=0x5 TLS -M- ...]
sent [EAP Response id=0x5 TLS Ack]
rcvd [EAP Request id=0x6 TLS --- ...]
 <- SSL/TLS Header: TLS 1.2
 <- Handshake: Server Hello
 <- SSL/TLS Header: TLS 1.2
 <- SSL/TLS Header: TLS 1.2
 <- InnerContentType (TLS1.3)
 <- Handshake: Encryped Extensions
 <- SSL/TLS Header: TLS 1.2
 <- InnerContentType (TLS1.3)
 <- Handshake: Certificate Request
 <- SSL/TLS Header: TLS 1.2
 <- InnerContentType (TLS1.3)
 <- Handshake: Certificate
certificate verify depth: 2
certificate verify depth: 1
certificate verify depth: 0
Peer name not specified: no check
 <- SSL/TLS Header: TLS 1.2
 <- InnerContentType (TLS1.3)
 <- Handshake: Certificate Verify
 <- SSL/TLS Header: TLS 1.2
 <- InnerContentType (TLS1.3)
 <- Handshake: Finished: TLS 1.3 (experimental)
 -> SSL/TLS Header: TLS 1.2
 -> ChangeCipherSpec
 -> SSL/TLS Header: TLS 1.2
 -> InnerContentType (TLS1.3)
 -> Handshake: Certificate
 -> SSL/TLS Header: TLS 1.2
 -> InnerContentType (TLS1.3)
 -> Handshake: Certificate Verify
 -> SSL/TLS Header: TLS 1.2
 -> InnerContentType (TLS1.3)
 -> Handshake: Finished: TLS 1.3 (experimental)
sent [EAP Response id=0x6 TLS --- ...]
rcvd [EAP Request id=0x7 TLS --- ...]
 <- SSL/TLS Header: TLS 1.2
 <- InnerContentType (TLS1.3)
 <- Handshake: New Session Ticket
EAP-TLS: Post-Handshake New Session Ticket arrived:
 <- SSL/TLS Header: TLS 1.2
 <- InnerContentType (TLS1.3)
 <- Handshake: New Session Ticket
EAP-TLS: Post-Handshake New Session Ticket arrived:
EAP-TLS generating MPPE keys
EAP-TLS PRF label = EXPORTER_EAP_TLS_Key_Material
sent [EAP Response id=0x7 TLS Ack]
rcvd [EAP Success id=0x7]
EAP authentication succeeded

[Evengard]

And the same flow from debug logs of SoftEther:

Request EAP
Got LCP packet request ID=2 OptionsListSize=0
ACKing empty LCP options list, id=2
Setting BEFORE_AUTH from ACK on LCP response parse on EAP accept
EAP: username=test, hubname=VPN
Got EAP-TLS size=280
Sent EAP-TLS size=1492 type=13 flag=192
Got EAP-TLS size=1
Sent EAP-TLS size=1492 type=13 flag=64
Got EAP-TLS size=1
Sent EAP-TLS size=1492 type=13 flag=64
Got EAP-TLS size=1
Sent EAP-TLS size=277 type=13 flag=0
Got EAP-TLS size=1230
SslCertVerifyCallback preverify error: 'unable to get local issuer certificate'
Sent EAP-TLS size=160 type=13 flag=0
Got EAP-TLS size=1
Accepted.
<snip>
NewIPC() Succeed.
LOG_THREAD: MsSetThreadPriorityIdle
Sent EAP-TLS size=0 SUCCESS
PPP auth success, ready for network layer on next tick

[Evengard]

If you compare the flow from PPPD and from Win11 SSTP, it is mostly the same. Except that instead of an ACK from the client on Windows, it sends access_denied TLS alert message and errors out on the client side with "The Local Security Authority cannot be contacted". Feels like the Windows client fails some weird checks. I wonder what they are...

[Evengard]

A TLS client handshake completed successfully. The negotiated cryptographic parameters are as follows.

   Protocol version: unknown
   CipherSuite: 0x1302
   Exchange strength: 0 bits
   Context handle: 0x1f97944a870
   Target name: 
   Local certificate subject name: CN=P2SChildCert
   Remote certificate subject name: CN=localhost

That's from SChannel ETW tracing. The "Protocol version: unknown" is kinda weird here. Shouldn't it be TLS 1.3 here? Maybe that's the reason for that cryptic error? More questions than answers...

[Evengard]

EapHostPeerGetResult returned a failure.
Eap Method Friendly Name: Microsoft: Smart Card or other certificate (EAP-TLS)
Reason code: 2148074244
Root Cause String: The Local Security Authority cannot be contacted

Repair String: Contact your network administrator for further assistance

That's another find. Apparently we got here an API call which actually returns the error, but the error itself... Converting it to HEX it gives us 0x80090304, and based on
https://learn.microsoft.com/en-us/windows/win32/com/com-error-codes-4 (the actual error message is the same) just SEC_E_INTERNAL_ERROR! Some internal error somewhere, without any details.

[Evengard]

Disregard all that rambling above, using the weird goto actually seems to fix it all up. I really need to reevaluate everything...

[domosekai]

I tested on two Win 11 computers and they both worked fine. Not domain-joined. One of them was recently fresh installed. (10.0.22621.1105)
I see your remote certificate is named localhost. Were you running the server on the localhost? Could that be the problem?

[Evengard]

No. The problem is that in my case I actually tried to complete the handshake, which for some reason works on TLS 1.2 but bugs on 1.3 on Win. In your case you abort the handshake as soon as you got the client certificate. I'll evaluate this and test with different clients, as I have a feeling this method may bug on them.

Technically it shouldn't (there's enough info for both server and client to authenticate each other), but different clients may have different expectations about it...

The localhost in the logs was just one of the examples, I had a server with a valid domainname and letsencrypt server certificate, that didn't change anything.

[Evengard]

I have bad news on that issue about the TLS 1.3 support for EAP-TLS. See, as I feared, the Windows client and PPPD under Linux does have different expectations about the TLS handshake flow. I've actually managed to decrypt the full flow, and I know where the problem lies.

The Windows client actually is ready to go as soon as the minimum amount of information for EAP-TLS completion was sent - namely, when it sends the client certificate (no matter which TLS protocol actually) without even completing the actual TLS handshake - it is already accepting EAP Success message, which results in a successful authentication.
It allows finishing the TLS handshake though, but in the case of TLS 1.3 only up until both the server and the client sent the TLS Finished message. Everything sent after that - namely, the Session Tickets - which are usually being sent post-handshake - seems to break it - with that cryptic "The Local Security Authority cannot be contacted" error and "access_denied" TLS alert.

PPPD instead is pretty strict about TLS handshakes. It expects the TLS handshake to be successfully completed until it can even accept an EAP Success message at all. Even more, for TLS 1.3, PPPD actually treats the Session Tickets as part of the handshake itself, refusint EAP Success until it have received it!

I guess you see the problem here: Windows breaks with Session Tickets, while PPPD can't work without them. Inherent incompatibility.

I see 4 solutions here:

1. we implement some sort of heuristics, trying to send EAP Success Windows style, seeing that the client didn't accept it, and finishing the handshake with the Session Tickets and resending EAP Success once again
2. we try to somehow renegotiate the EAP flow from the start if the Session Tickets method failed, falling back to the Windows style (not sure the Windows client will even accept that)
3. we ignore the fact that PPPD's TLS 1.3 code is broken (it defaults to 1.2 anyway) and implement the Windows-style authentication only disabling sending of theese session tickets (which shouldn't affect the TLS 1.2 PPPD flow)
4. or we just disable TLS 1.2 for now until PPPD fixes their experimental code (pls refer to EAP-TLS 1.3 client is expecting session_tickets when it shouldn't ppp-project/ppp#388)

[Evengard]

VPN Client Pro on Android with EAP-TLS 1.3 is broken completely, with no way to fix it, the method to "jump start" it no longer works it seems, and there's no way to actually force it to stay on 1.2 at all. Honestly, while I do have a working implementation for 1.3 which deals OK with Windows on both 1.3 and 1.2, deals OK with PPPD in 1.2 mode, all the other older clients... I have no ideas what to do with them. Just ditch them? Or maybe make that "force TLS 1.2 for EAP-TLS" a configuration parameter?

[Evengard]

I've updated the pull request, more info there (#1751)