2

I've a working ValidatingWebhookConfiguration and have been creating|approving CSRs with certificates.k8s.io/v1beta.

I upgraded (MicroK8s) from 1.18 to 1.20 and received a warning that certificates.k8s.io/v1beta is deprecated 1.19+ and thought I'd try (without success) upgrading to certificates.k8s.io/v1.

Existing (working) CSR:

apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
  name: ${SERVICE}.${NAMESPACE}
spec:
  groups:
  - system:authenticated
  request: $(cat ${FILENAME}.csr | base64 | tr -d '\n')
  usages:
  - digital signature
  - key encipherment
  - server auth

Upgrading the API generated an error:

missing required field "signerName" in io.k8s.api.certificates.v1.CertificateSigningRequestSpec;

I read the CSR doc specifically the part about Kubernetes signers and, because my existing spec uses server auth, assumed I could use kubernetes.io/kubelet-serving since this is the only one that permits server auth.

apiVersion: certificates.k8s.io/v1                      <<--- UPGRADED
kind: CertificateSigningRequest
metadata:
  name: ${SERVICE}.${NAMESPACE}
spec:
  groups:
  - system:authenticated
  request: $(cat ${FILENAME}.csr | base64 | tr -d '\n')
  signerName: kubernetes.io/kubelet-serving              <<--- ADDED
  usages:
  - digital signature
  - key encipherment
  - server auth

However, I get errors trying to approve the CSR (as a cluster admin):

kubectl certificate approve ${SERVICE}.${NAMESPACE}
certificatesigningrequest.certificates.k8s.io/${SERVICE}.${NAMESPACE} approved

kubectl get csr ${SERVICE}.${NAMESPACE}
NAME                      SIGNERNAME                      REQUESTOR   CONDITION
${SERVICE}.${NAMESPACE}   kubernetes.io/kubelet-serving   admin       Approved,Failed

NOTE Approved but Failed

And I'm unable to get the certificate (presumably because it Failed):

kubectl get csr ${SERVICE}.${NAMESPACE} \
--output=jsonpath='{.status.certificate}'

How should I use the certificates.k8s.io/v1 API?

Update: 2021-01-06

OK, so I realized I have more information on the "Failed" and this gives me something to investigate...

kubectl get csr/${SERVICE}.${NAMESPACE} \
--output=jsonpath="{.status}" \
| jq .

Yields:

{
  "conditions": [
    {
      "lastTransitionTime": "2021-01-06T18:52:15Z",
      "lastUpdateTime": "2021-01-06T18:52:15Z",
      "message": "This CSR was approved by kubectl certificate approve.",
      "reason": "KubectlApprove",
      "status": "True",
      "type": "Approved"
    },
    {
      "lastTransitionTime": "2021-01-06T18:52:15Z",
      "lastUpdateTime": "2021-01-06T18:52:15Z",
      "message": "subject organization is not system:nodes",
      "reason": "SignerValidationFailure",
      "status": "True",
      "type": "Failed"
    }
  ]
}

Update: 2021-01-07

Thanks @PjoterS

ubectl describe csr/${SERVICE}.${NAMESPACE}
Name:         eldlund.utopial
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"certificates.k8s.io/v1","kind":"CertificateSigningRequest","metadata":{"annotations":{},"name":"eldlund.utopial"},"spec":{"groups":["system:authenticated"],"request":"LS0tLS1C...LS0tLS0K","signerName":"kubernetes.io/kubelet-serving","usages":["digital signature","key encipherment","server auth"]}}

CreationTimestamp:  Thu, 07 Jan 2021 17:03:23 +0000
Requesting User:    admin
Signer:             kubernetes.io/kubelet-serving
Status:             Pending
Subject:
  Common Name:    eldlund.utopial.svc
  Serial Number:  
Subject Alternative Names:
         DNS Names:  eldlund.utopial.svc
                     eldlund.utopial.svc.cluster.local
Events:  <none>

Signing w/ OpenSSL (rather than Kubernetes)

I tried creating CA crt|key and then a service key|CSR and signing the service CSR with the CA but Kubernetes complains:

x509: certificate is not valid for any names, but wanted to match ainsley.utopial.svc

Yet the certificate appears to contain both CN and SAN entries:

DOESN'T WORK

openssl x509 -in ${FILENAME}.crt --noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            6f:14:25:8c:...
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = Validating Webhook CA
        Validity
            Not Before: Jan  7 18:10:50 2021 GMT
            Not After : Feb  6 18:10:50 2021 GMT
        Subject: CN = ainsley.utopial.svc
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:ca:56:15:...
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                DNS:ainsley.utopial.svc, DNS:ainsley.utopial.svc.cluster.local
    Signature Algorithm: sha256WithRSAEncryption
         b2:ec:22:b6:...

NOTE CN is a DNS name above but an IP below ???

Reverting to my working solution with v1beta1 and changing the service name for completeness (loi), the Webhook succeeds and the certificate appears to be no different from the one shown above (except the different service name):

WORKS

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ff:b3:cb:11:...
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = 10.152.183.1
        Validity
            Not Before: Jan  7 18:18:45 2021 GMT
            Not After : Jan  7 18:18:45 2022 GMT
        Subject: CN = loi.utopial.svc
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:d2:cc:c2:...
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier: 
                keyid:E7:AE:3A:25:95:D2:F7:5B:C6:EA:50:56:07:E8:25:83:60:88:68:7A

            X509v3 Subject Alternative Name: 
                DNS:loi.utopial.svc, DNS:loi.utopial.svc.cluster.local
    Signature Algorithm: sha256WithRSAEncryption
         48:a1:b2:e2:...
3
  • Could you provide describe output of your CSR kubectl describe csr <csrName>? When you are downloading your CSR are you getting output or its empty?
    – PjoterS
    Jan 7, 2021 at 7:38
  • Thanks for replying, @pjoters. I've updated my question with the describe results.
    – DazWilkin
    Jan 7, 2021 at 17:06
  • I discovered this link last night and it may hold my solution. It doesn't use Kubernetes for CSR. Checking whether it works now
    – DazWilkin
    Jan 7, 2021 at 17:21

3 Answers 3

1

Update Switched to cert-manager and everything working well

I got it working but I'm unsure why what I'm now doing is correct.

And the openssl feels unwieldy (advice appreciated).

Environment

DIR=${PWD}/secrets
SERVICE="..."
NAMESPACE="..."

FILENAME="${DIR}/${SERVICE}.${NAMESPACE}"

CA

openssl req \
-nodes \
-new \
-x509 \
-keyout ${FILENAME}.ca.key \
-out ${FILENAME}.ca.crt \
-subj "/CN=Validating Webhook CA"

Create (Webhook) Service

Necessary to set the service certificate's CN to the IP

cat ./kubernetes/service.yaml \
| sed "s|SERVICE|${SERVICE}|g" \
| sed "s|NAMESPACE|${NAMESPACE}|g" \
| kubectl apply --filename=- --namespace=${NAMESPACE}

ENDPOINT=$(\
  kubectl get service/${SERVICE} \
  --namespace=${NAMESPACE} \
  --output=jsonpath="{.spec.clusterIP}") && echo ${ENDPOINT}

Create CSR

Even though I include the CN and alt_names here, I must duplicate the SAN stuff (next step)

echo "[ req ]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn
req_extensions = req_ext

[ dn ]
commonName = ${ENDPOINT}

[ req_ext ]
subjectAltName = @alt_names

[alt_names]
DNS.1 = ${SERVICE}.${NAMESPACE}.svc
DNS.2 = ${SERVICE}.${NAMESPACE}.svc.cluster.local
" > ${FILENAME}.cfg

openssl req \
-nodes \
-new \
-sha256 \
-newkey rsa:2048 \
-keyout ${FILENAME}.key \
-out ${FILENAME}.csr \
-config ${FILENAME}.cfg

Create CSR extension

Unsure why I must duplicate (or separate) this content. If I omit this from the openssl x509 -extfile, the certificate contains no SAN extension.

printf "subjectAltName=DNS:${SERVICE}.${NAMESPACE}.svc,DNS:${SERVICE}.${NAMESPACE}.svc.cluster.local" > ${FILENAME}.ext

Create service certificate

How can I use a single CSR for everything rather than CSR+EXT?

openssl x509 \
-req \
-in ${FILENAME}.csr \
-extfile ${FILENAME}.ext \
-CA ${FILENAME}.ca.crt \
-CAkey ${FILENAME}.ca.key \
-CAcreateserial \
-out ${FILENAME}.crt

Create (Webhook) Deployment

The underlying implementation of the webhook needs the service's crt|key

kubectl create secret tls ${SERVICE} \
--namespace=${NAMESPACE} \
--cert=${FILENAME}.crt \
--key=${FILENAME}.key

cat ./kubernetes/deployment.yaml \
| sed "s|SERVICE|${SERVICE}|g" \
| sed "s|NAMESPACE|${NAMESPACE}|g" \
| kubectl apply --filename=- --namespace=${NAMESPACE}

Create Webhook

Grab the CA certificate

CABUNDLE=$(openssl base64 -A <"${FILENAME}.ca.crt")

cat ./kubernetes/webhook.yaml \
| sed "s|SERVICE|${SERVICE}|g" \
| sed "s|NAMESPACE|${NAMESPACE}|g" \
| sed "s|CABUNDLE|${CABUNDLE}|g" \
| kubectl apply --filename=- --namespace=${NAMESPACE}
0

FYI, by mocking your apps CSR's with kubelet-like requests, you can continue to have the k8s certificates created. I have the details in this issue, https://github.com/kubernetes/website/issues/26111, trying to get the k8s docs to be more useful on this topic.

1
  • Thanks for the ping. The documentation isn't sufficiently generalized in this regard. I'd assumed that this process (i.e. for non-core resources) had become discouraged. Thanks!
    – DazWilkin
    Jan 25, 2021 at 20:09
0

While generating .csr from OpenSSL you can add "system:node" and "O=system:nodes" in CN(i.e. common name) as below and then use your upgraded certificates.k8s.io/v1.

Before change OpenSSL command openssl req -new -key ${tmpdir}/server-key.pem -subj "/CN=${title}.${namespace}.svc" -out ${tmpdir}/server.csr -config ${tmpdir}/csr.conf

After change OpenSSL command openssl req -new -key ${tmpdir}/server-key.pem -subj "/CN=system:node:${title}.${namespace}.svc;/O=system:nodes" -out ${tmpdir}/server.csr -config ${tmpdir}/csr.conf

1
  • Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – Community Bot
    Dec 17, 2022 at 12:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.