103

I am aware that Let's Encrypt made changes that may impact older clients because a root certificate would expire. See DST Root CA X3 Expiration (September 2021).

However, I didn't think this could impact me because my development machine is up-to-date.

But since today I get the message while doing a git pull:

fatal: unable to access 'https://git.company.tld/project.git/': SSL certificate problem: certificate has expired

I just downloaded the newest Git for Windows (2.33.0) and confirmed that the built-in OpenSSL is up-to-date (OpenSSL 1.1.1k 25 Mar 2021) which should be good.

OpenSSL Client Compatibility Changes for Let’s Encrypt Certificates

But the error seems to stay.

openssl s_client -showcerts -connect git.company.tld:443

shows

CONNECTED(000001A0)
---
Certificate chain
 0 s:CN = git.company.tld
   i:C = US, O = Let's Encrypt, CN = R3
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
---
Server certificate
subject=CN = git.company.tld

issuer=C = US, O = Let's Encrypt, CN = R3

---
No client certificate CA names sent
Peer signing digest: SHA512
Peer signature type: RSA
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 3058 bytes and written 443 bytes
Verification error: certificate has expired
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: ...
    Session-ID-ctx:
    Master-Key: ...
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1632982992
    Timeout   : 7200 (sec)
    Verify return code: 10 (certificate has expired)
    Extended master secret: no
---

The problem is not with the issued certificate itself which is not expired and accepted by Chrome (Windows certificate store) and Firefox.

4

12 Answers 12

63

On my Ubuntu 16.04.6 LTS (Xenial Xerus) machine, the solution was to remove the DST Root CA X3 certificate by running the following two commands:

sudo rm /usr/share/ca-certificates/mozilla/DST_Root_CA_X3.crt
sudo update-ca-certificates

On my Mac OS X 10.13.6 (High Sierra) machine, cURL (and therefore Git) rely on the /etc/ssl/cert.pem file for root CA verification.

The solution was to remove the DST Root CA X3 certificate, which expired today, from the file:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            44:af:b0:80:d6:a3:27:ba:89:30:39:86:2e:f8:40:6b
    Signature Algorithm: sha1WithRSAEncryption
        Issuer: O=Digital Signature Trust Co., CN=DST Root CA X3
        Validity
            Not Before: Sep 30 21:12:19 2000 GMT
            Not After : Sep 30 14:01:15 2021 GMT
        Subject: O=Digital Signature Trust Co., CN=DST Root CA X3
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:df:af:e9:97:50:08:83:57:b4:cc:62:65:f6:90:
                    82:ec:c7:d3:2c:6b:30:ca:5b:ec:d9:c3:7d:c7:40:
                    c1:18:14:8b:e0:e8:33:76:49:2a:e3:3f:21:49:93:
                    ac:4e:0e:af:3e:48:cb:65:ee:fc:d3:21:0f:65:d2:
                    2a:d9:32:8f:8c:e5:f7:77:b0:12:7b:b5:95:c0:89:
                    a3:a9:ba:ed:73:2e:7a:0c:06:32:83:a2:7e:8a:14:
                    30:cd:11:a0:e1:2a:38:b9:79:0a:31:fd:50:bd:80:
                    65:df:b7:51:63:83:c8:e2:88:61:ea:4b:61:81:ec:
                    52:6b:b9:a2:e2:4b:1a:28:9f:48:a3:9e:0c:da:09:
                    8e:3e:17:2e:1e:dd:20:df:5b:c6:2a:8a:ab:2e:bd:
                    70:ad:c5:0b:1a:25:90:74:72:c5:7b:6a:ab:34:d6:
                    30:89:ff:e5:68:13:7b:54:0b:c8:d6:ae:ec:5a:9c:
                    92:1e:3d:64:b3:8c:c6:df:bf:c9:41:70:ec:16:72:
                    d5:26:ec:38:55:39:43:d0:fc:fd:18:5c:40:f1:97:
                    eb:d5:9a:9b:8d:1d:ba:da:25:b9:c6:d8:df:c1:15:
                    02:3a:ab:da:6e:f1:3e:2e:f5:5c:08:9c:3c:d6:83:
                    69:e4:10:9b:19:2a:b6:29:57:e3:e5:3d:9b:9f:f0:
                    02:5d
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
            X509v3 Subject Key Identifier:
                C4:A7:B1:A4:7B:2C:71:FA:DB:E1:4B:90:75:FF:C4:15:60:85:89:10
    Signature Algorithm: sha1WithRSAEncryption
         a3:1a:2c:9b:17:00:5c:a9:1e:ee:28:66:37:3a:bf:83:c7:3f:
         4b:c3:09:a0:95:20:5d:e3:d9:59:44:d2:3e:0d:3e:bd:8a:4b:
         a0:74:1f:ce:10:82:9c:74:1a:1d:7e:98:1a:dd:cb:13:4b:b3:
         20:44:e4:91:e9:cc:fc:7d:a5:db:6a:e5:fe:e6:fd:e0:4e:dd:
         b7:00:3a:b5:70:49:af:f2:e5:eb:02:f1:d1:02:8b:19:cb:94:
         3a:5e:48:c4:18:1e:58:19:5f:1e:02:5a:f0:0c:f1:b1:ad:a9:
         dc:59:86:8b:6e:e9:91:f5:86:ca:fa:b9:66:33:aa:59:5b:ce:
         e2:a7:16:73:47:cb:2b:cc:99:b0:37:48:cf:e3:56:4b:f5:cf:
         0f:0c:72:32:87:c6:f0:44:bb:53:72:6d:43:f5:26:48:9a:52:
         67:b7:58:ab:fe:67:76:71:78:db:0d:a2:56:14:13:39:24:31:
         85:a2:a8:02:5a:30:47:e1:dd:50:07:bc:02:09:90:00:eb:64:
         63:60:9b:16:bc:88:c9:12:e6:d2:7d:91:8b:f9:3d:32:8d:65:
         b4:e9:7c:b1:57:76:ea:c5:b6:28:39:bf:15:65:1c:c8:f6:77:
         96:6a:0a:8d:77:0b:d8:91:0b:04:8e:07:db:29:b6:0a:ee:9d:
         82:35:35:10
-----BEGIN CERTIFICATE-----
MIIDSjCCAjKgAwIBAgIQRK+wgNajJ7qJMDmGLvhAazANBgkqhkiG9w0BAQUFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTAwMDkzMDIxMTIxOVoXDTIxMDkzMDE0MDExNVow
PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD
Ew5EU1QgUm9vdCBDQSBYMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AN+v6ZdQCINXtMxiZfaQguzH0yxrMMpb7NnDfcdAwRgUi+DoM3ZJKuM/IUmTrE4O
rz5Iy2Xu/NMhD2XSKtkyj4zl93ewEnu1lcCJo6m67XMuegwGMoOifooUMM0RoOEq
OLl5CjH9UL2AZd+3UWODyOKIYepLYYHsUmu5ouJLGiifSKOeDNoJjj4XLh7dIN9b
xiqKqy69cK3FCxolkHRyxXtqqzTWMIn/5WgTe1QLyNau7Fqckh49ZLOMxt+/yUFw
7BZy1SbsOFU5Q9D8/RhcQPGX69Wam40dutolucbY38EVAjqr2m7xPi71XAicPNaD
aeQQmxkqtilX4+U9m5/wAl0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNV
HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFMSnsaR7LHH62+FLkHX/xBVghYkQMA0GCSqG
SIb3DQEBBQUAA4IBAQCjGiybFwBcqR7uKGY3Or+Dxz9LwwmglSBd49lZRNI+DT69
ikugdB/OEIKcdBodfpga3csTS7MgROSR6cz8faXbauX+5v3gTt23ADq1cEmv8uXr
AvHRAosZy5Q6XkjEGB5YGV8eAlrwDPGxrancWYaLbumR9YbK+rlmM6pZW87ipxZz
R8srzJmwN0jP41ZL9c8PDHIyh8bwRLtTcm1D9SZImlJnt1ir/md2cXjbDaJWFBM5
JDGFoqgCWjBH4d1QB7wCCZAA62RjYJsWvIjJEubSfZGL+T0yjWW06XyxV3bqxbYo
Ob8VZRzI9neWagqNdwvYkQsEjgfbKbYK7p2CNTUQ
-----END CERTIFICATE-----

After removing the entire code snippet above from the file and saving it, the error went away.

3
  • I'm on 16.04.2 and removing that file + updating didn't help. I cannot curl anywhere with SSL..
    – Shautieh
    Commented Oct 4, 2021 at 17:14
  • Thx, it works also on my macO 10.15
    – mmerle
    Commented Oct 8, 2021 at 12:33
  • this actually works for my mac, thanks
    – zer0
    Commented Feb 4, 2022 at 5:51
19

I had the same issue because I was running an old version of Git for Windows (2.15.0). After updating Git to the latest version (2.33) the problem was fixed.

Reason: Older versions of Git would not accept the expired root certificate from Let's Encrypt.

15

I faced the same problem on an Ubuntu 14.04 LTS (Trusty Tahr) server.

This problem affected cURL calls from PHP, etc.

Example: from a simple curl on the command line,

curl https://curl.se/ca/cacert.pem

I was getting the certificate expired message. It was due to the old Let's Encrypt certificate expiration. dst-root-ca-x3-expiration-september-2021

So, here is a simple working solution:

Edit the file /etc/ca-certificates.conf.

Find and comment the lines with ! like this:

!mozilla/DST_Root_CA_X3.crt

Or run the following command to automatically comment the certificate:

sudo sed -i -e 's/mozilla\/DST_Root_CA_X3\.crt/!mozilla\/DST_Root_CA_X3\.crt/g' /etc/ca-certificates.conf

Save the file

Update certificates:

sudo update-ca-certificates

And all is done... It's working, and there isn't any need to change other things...

7
  • Hi, thanks, does this solution have any disadvantages for a website to which visitors connect? Commented Oct 1, 2021 at 17:55
  • @christian audebert thank you very much, you saved my headache. I was so close I firstly tried manually remove /etc/ssl/certs/DST_Root_CA_X3.crt but did not help and I reverted it back and secondly deleted it from /etc/ssl/ca-certificates.crt but it did not help too (my bad, I had to do both steps to solve issue for testing). Of course correct way is to edit /etc/ca-certificates.conf but I wanted conservatively do quick test if it helps without doing update-ca-certificates. Now curl via terminal and curl via PHP scripts can access lets encrypt-ed websites :)
    – mikep
    Commented Oct 1, 2021 at 18:23
  • @user2267379 this does not affect website visitors, this change is only for server which acts as client when connecting to HTTPS website (e.g. using curl, wget, etc.)
    – mikep
    Commented Oct 1, 2021 at 18:26
  • @mekip : some visitors can no longer connect to the site since the change, which solution works for them? Commented Oct 2, 2021 at 17:10
  • Tried the same with ubuntu 16.04.2 to no avail. Would you have another idea? I've tried so many things but curl won't run correectly anymore..
    – Shautieh
    Commented Oct 4, 2021 at 17:19
13

On our Windows test clients we had to update Git to the latest version.

Since Git 2.16.1(2) you can use

git update-git-for-windows

In version between 2.14.2 and 2.16.1, the command was

git update

See also: How to upgrade Git on Windows to the latest version

6

For those who have servers running on Ubuntu, with Certbot managing certificates, I have forced the renewals using ISRG Root X1. This way, new certificates don't contain the chain of DST Root CA X3, and this did the trick for us.

To do that, first check if your Certbot version is < 1:

sudo certbot --version

If so, you have to remove it and reinstall using Snap:

sudo apt-get remove -y certbot python3-certbot-apache
sudo snap install certbot --classic
sudo ln -s /snap/bin/certbot /usr/bin/certbot

After reinstalling, or if your Certbot version is > 1, force the renewal:

sudo certbot renew --force-renewal --preferred-chain "ISRG Root X1"

I also have used DigiCert SSL Installation Diagnostics Tool to check my certificates, before and after renewing, to verify if the DST X3 chain was removed.

1
  • 1
    Thanks a lot! Worked perfectly for my ubuntu 14 apache server. curl error (error 60) is now gone.
    – paperboy
    Commented Oct 3, 2021 at 2:41
4

Apparently this is not a client issue, but the Let's Encrypt certificate being served by a Sophos UTM WAF (latest version, 9.707-5).

The same certificate served from an Apache web server works fine (and the openssl s_client -showcerts response looks different -> more entries in the certificate chain). So this is not a client-related problem.

Update

The solution was to delete the old Lets Encrypt CA (48:50:4E:97:...)

CA

from Webserver ProtectionCertificate ManagementCertificate Authority.

Client requests worked instantly.

Thanks to VolkerZier in the Sophos forum for giving the hint. See (in German) Let's Encrypt Root Zertifikat gültig bis 30.09.2021 (alte R3 / X3 Zertifikatskette).

4

For applications based on OpenSSL <= 1.0.2 such as Ubuntu 12.04 (Precise Pangolin), you need to allow OpenSSL to use the alternate chain path to trust the remote site.

First you need to install the ISRG_Root_X1.crt certificate and remove the expired one from the trusted store: DST_Root_CA_X3.crt

This will allow that clients using OpenSSL like Wget, cURL, etc. to work again. The steps are:

To install the ISRG_Root_X1 certificate:

sudo wget http://launchpadlibrarian.net/482351465/ca-certificates_20190110~12.04.1_all.deb
sudo dpkg -i ca-certificates_20190110~12.04.1_all.deb

To disable the DST_Root_CA_X3 certificate:

sudo vi /etc/ca-certificates.conf

Then set: !mozilla/DST_Root_CA_X3.crt

Note: In this file, when the line begins with # is comment. When the line begins with ! is certificate filename to be deselected.

And finally run:

sudo update-ca-certificates

After that, wget and curl will not complain any more.

The best explanation I've found out there is the video DST Root CAX3 Expiration Sept 2021 (34 minutes).

3
  • What is "!"? A comment character? To outcomment (disable)? Can you clarify? Please respond by editing (changing) your answer, not here in comments (without "Edit:", "Update:", or similar - the answer should appear as if it was written today). Commented Oct 17, 2021 at 14:09
  • so change it to #!mozilla/DST_Root_CA_X3.crt ??
    – Shaho
    Commented Oct 19, 2021 at 13:03
  • No. Just "!" without "#" cause the cert will be deselected.
    – frommelmak
    Commented Oct 22, 2021 at 6:08
2

I was facing a similar issue with DevOps build agents. But I can access the DevOps server web interface without any issue.

To solve this,

  • I updated my Let's Encrypt client (I'm using Certify The Web)
  • I have renewed my certificate

After that, the DevOps agent is able to do a Git pull.

2

Verify the version of GnuTLS (libgnutls). The issue with alternate chains was fixed in 3.6.13-4. So updating GnuTLS to a version above this might solve the issue for Git.

Backstory:

(On Debian systems at least) curl/wget uses libssl/OpenSSL and Git uses libgnutls30 via libcurl3-gnutls.

0

I (as will many in the Third World), have several Windows XP 32/64-bit machines. They are since Friday giving me the "YOUR CLOCK IS AHEAD" error, which is part, I believe, of this SSL certificate lapse. (They were all flawless, universally across the web with Chrome previously).

Many websites (~40%) I visit on the Windows XP machines (handy for legacy software, etc), all give the same TIME error-msg.

My instinct tells me:

Those new certificates have to be acquired, installed and become active and that's going to TAKE TIME to propagate system-wide, meaning ... globally.

0

The same issue here. For me, it helps to update Visual Studio 2017.

2
  • Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – Community Bot
    Commented Oct 5, 2021 at 12:52
  • 1
    "update Visual Studio 2017" or "update to Visual Studio 2017" (you have already left out a lot of words)? If the latter, from which version (e.g., Visual Studio 2005)? Even if it is patching the installed version, can you make it clearer (e.g. how was it updated and to which exact version number)? Please respond by editing (changing) your answer, not here in comments (without "Edit:", "Update:", or similar - the answer should appear as if it was written today). Commented Oct 17, 2021 at 14:29
0

curl <url> -X POST -H 'authorization: Bearer xxx' [other option]

curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

curl <url> -X POST -H 'authorization: Bearer xxx' [other option] -k

-k, --insecure

(TLS SFTP SCP) By default, every secure connection curl makes is verified to be secure before the transfer takes place. This option makes curl skip the verification step and proceed without checking. curl/manpage

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.