14

Was trying to restrict IAM users with the rbac of AWS EKS cluster. Mistakenly updated the configmap "aws-auth" from kube-system namespace. This removed the complete access to the EKS cluster.

Missed to add the groups: in the configmap for the user.

Tried providing full admin access to the user/role that is lastly mentioned in the configmap, But no luck.

Any idea of recovering access to the cluster would be highly appreciable.

The config-map.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapUsers: |
    - userarn: arn:aws:iam::1234567:user/test-user
      username: test-user

2 Answers 2

13

Did a work-around for this issue:

Since the IAM user who created the EKS Cluster by default possess complete access over the cluster, inspite of the aws-auth configmap. Since the IAM user who created, had been deleted, we re-created the IAM user, as it would have the same arn (if IAM user is created with the same name as before).

Once created the user credentials(access & secret keys) for the user, we got back access to the EKS cluster. Following which, we modified the config-map as required.

0
0

First thing I would try is restoring the original aws-auth ConfigMap (you can find it here):

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    - rolearn: <ARN of instance role (not instance profile)>
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes

Replace the placeholder for rolearn with the ARN of the IAM role associated with your worker nodes as explained in the EKS documentation.

When the cluster works again, you can add IAM users to the ConfigMap again, which is also described in the EKS docs.

3
  • 5
    How can I modify the configMap as I lost access to the cluster in the first place? Facing the same issue. Please suggest. Apr 28, 2020 at 5:09
  • One possibility is if one of your Pods uses a ServiceAccount that has permissions to modify ConfigMaps: log in to the node where the Pod is running with SSH, exec into the Pod's Docker container (docker exec -ti <container> bash), and retrieve the ServiceAccount token in /var/run/secrets/kubernetes.io/serviceaccount/token. You can then use this token to access the cluster from your local machine with kubectl.
    – weibeld
    Apr 28, 2020 at 9:24
  • @AnilBhargava Sorry for the late response. If you are the IAM User who created the user, even if you deleted the config map by mistake, your IAM user has the root access yet. That way, you can use the iam arn & access,secret key, you can gain acccess on the cluster back.
    – Sandy
    Dec 15, 2020 at 13:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.