Wrong SSL certificate on mirrorlists.choria.io

[pjakuszew]

It seems that mirrorlists.choria.io is using wrong SSL certificate - both apt and openssl s_client report that there's a CN mismatch:

$ openssl s_client -connect mirrorlists.choria.io:443 -servername mirrorlists.choria.io
CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = apt.uk.choria.io
verify return:1
---
Certificate chain
 0 s:CN = apt.uk.choria.io
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
...

Output from apt:

# apt update
...
Get:11 http://mirrorlists.choria.io/apt/release/ubuntu/bionic/mirrors.txt Mirrorlist [144 B]
Ign:12 https://apt.uk.choria.io/release/ubuntu/bionic ubuntu InRelease
Ign:12 https://apt.us.choria.io/release/ubuntu/bionic ubuntu InRelease
Ign:12 https://apt.eu.choria.io/release/ubuntu/bionic ubuntu InRelease
Ign:13 https://apt.us.choria.io/release/ubuntu/bionic ubuntu Release
Ign:13 https://apt.eu.choria.io/release/ubuntu/bionic ubuntu Release
Err:13 https://apt.uk.choria.io/release/ubuntu/bionic ubuntu Release
  Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate.  Could not handshake: Error in the certificate verification. [IP: 66.228.41.18 443]
Reading package lists... Done
E: The repository 'mirror://mirrorlists.choria.io/apt/release/ubuntu/bionic/mirrors.txt ubuntu Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

[ripienaar]

hmm, mirrorlists should be accessed over http not https, apt has mirror+https and mirror protocols, so not sure how its hitting this.

What debian version is this? bionic, ok, not sure wtf is going on with debian people, they just change how things work every morning :)

[pjakuszew]

This is from Ubuntu 18.04 (bionic). curl output is from Ubuntu 20.04 (focal).

[pjakuszew]

Indeed apt reaches mirrorlists via http, it seems that it doesn't like the cert from some returned mirror...

[ripienaar]

OK, so that then claims, I guess, apt.us.choria.io is expired based on the IP it shows - but that doesnt appear to be the case

[pjakuszew]

Here is what happens when I specify the EU mirror manually:

Ign:11 https://apt.eu.choria.io/release/ubuntu/bionic ubuntu InRelease
Err:12 https://apt.eu.choria.io/release/ubuntu/bionic ubuntu Release
  Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate.  Could not handshake: Error in the certificate verification. [IP: 128.199.57.137 443]
Reading package lists... Done
E: The repository 'https://apt.eu.choria.io/release/ubuntu/bionic ubuntu Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

[ripienaar]

I set up a container here with bionic and it works with mirror lists and all mirrors.

Do you have latest ca-certificates on your machine?

[ripienaar]

pretty sure this is somehow related to the recent lets encrypt expiry, if all your roots are up to date it probably works. I think to get to the bottom of it you'd have to walk the chain till you find whats dead

https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/

[pjakuszew]

Well, looks like it is an issue with that particular machine, then. I find it weird because I installed it from scratch like two hours ago... I will try to troubleshoot this further, thank you for checking that on your side.

[ripienaar]

OK, let me know what you find, presumably your clocks is fine too :) Best I can tell nothing in the chains are expired

[pjakuszew]

You were right, ca-certificates needed an update - it's just that the local APT mirror in my "test" environment had outdated packages. On that particular machine it was ca-certificates_20210119~18.04.1_all.deb, while the latest working one is ca-certificates_20210119~18.04.2_all.deb. I pulled that package manually and now apt refreshes its cache properly.

Thanks for pointing that out, I will go update my mirrors then :)

[ripienaar]

very good, glad thats sorted out :)