4

Hi there I was reviewing the GKE autopilot mode and noticed that in cluster configureation istio is disabled and I'm not able to change it. Also installation via istioctl install fail with following error

 error   installer       failed to update resource with server-side apply for obj MutatingWebhookConfiguration//istio-sidecar-injector: mutatingwebhookconfigurations.admissionregistration.k8s.io "istio-sidecar-injector" is forbidden: User "something@example" cannot patch resource "mutatingwebhookconfigurations" in API group "admissionregistration.k8s.io" at the cluster scope: GKEAutopilot authz: cluster scoped resource "mutatingwebhookconfigurations/" is managed and access is denied

Am I correct or it's not possible to run istio in GKE autopilot mode?

3 Answers 3

7

TL;DR

It is not possible at this moment to run istio in GKE autopilot mode.

Conclusion

If you are using Autopilot, you don't need to manage your nodes. You don't have to worry about operations such as updating, scaling or changing the operating system. However, the autopilot has a number of limitations.

Even if you are trying to install istio with a command istioctl install, istio will not be installed. You will then see the following message:

This will install the Istio profile into the cluster. Proceed? (y/N) y

✔ Istio core installed
✔ Istiod installed
✘ Ingress gateways encountered an error: failed to wait for resource: resources not ready after 5m0s: timed out waiting for the condition Deployment/istio-system/istio-ingressgateway

  • Pruning removed resources 2021-05-07T08:24:40.974253Z warn installer retrieving resources to prune type admissionregistration.k8s.io/v1beta1, Kind=MutatingWebhookConfiguration: mutatingwebhookconfigurations.admissionregistration.k8s.io is forbidden: User "something@example" cannot list resource "mutatingwebhookconfigurations" in API group "admissionregistration.k8s.io" at the cluster scope: GKEAutopilot authz: cluster scoped resource "mutatingwebhookconfigurations/" is managed and access is denied not found Error: failed to install manifests: errors occurred during operation

This command failed, bacuse for sidecar injection, installer tries to create a MutatingWebhookConfiguration called istio-sidecar-injector. This limitation is mentioned here.

For more information you can also read this page.

1
  • Thank you Mikołaj, this is what I thought just wanted to be 100% sure. May 7, 2021 at 9:10
2

It is not possible to create mutating admission webhooks according to documentation

You cannot create custom mutating admission webhooks for Autopilot clusters

Since Istio uses mutating webhooks to inject its sidecars, it will probably not work and it is also consistent with the error you get.

1
  • It's for security reason, no high privilege, no mutating webhook, no ssh into the node,... May 6, 2021 at 20:03
0

According to the documentation this should be possible with GKE 1.21:

In GKE version 1.21.3-gke.900 and later, you can create validating and mutating dynamic admission webhooks. However, Autopilot modifies the admission webhooks objects to add a namespace selector which excludes the resources in managed namespaces (currently, kube-system) from being intercepted. Additionally, webhooks which specify one or more of following resources (and any of their sub-resources) in the rules, will be rejected:

  • group: "" resource: nodes
  • group: certificates.k8s.io resource: certificatesigningrequests
  • group: authentication.k8s.io resource: tokenreviews

https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview#webhooks_limitations

1
  • Sadly it still wouldn't install because you are not allowed to set NET_ADMIN capability. To mitigate this I tried to install istio cni, but that needs hostnetwork which is also denied.
    – MrBlaise
    Oct 6, 2021 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.