How do I use AWS EKS with the Jenkins Kubernetes Cloud plugin?

Question

I have found essentially no documentation about how to use the Jenkins Kubernetes Plugin with Amazon EKS. The documentation mentions aws-iam-authenticator and a java setting to change a cache timeout, but doesn't explain how to configure anything.

If I put the API URL for my EKS cluster in the "Kubernetes URL" field and click the "Test Connection" button I get an error about the certification path:

Error testing connection https://XXX.gr7.us-west-2.eks.amazonaws.com: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

If I then check the "Disable https certificate check" checkbox I get an error about the "system:anonymous" user not having the right permissions:

Error testing connection https://XXX.gr7.us-west-2.eks.amazonaws.com: Failure executing: GET at: https://XXX.gr7.us-west-2.eks.amazonaws.com/api/v1/namespaces/default/pods. Message: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "default". Received status: Status(apiVersion=v1, code=403, details=StatusDetails(causes=[], group=null, kind=pods, name=null, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "default", metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Forbidden, status=Failure, additionalProperties={}).

The EC2 instance the jenkins master is running on has an IAM role with full eks:* permissions. If I set up a Credential with the IAM Role it doesn't show up in the Credentials dropdown. A Credential with an AWS access and secret key also doesn't show up in the Credentials dropdown.

Answer 1

Been there and resolved that.

If you've setup your kubeconfig to use aws eks get-token .., then ensure your Jenkins Master has AWS CLI version 1.16.300 and up which supports fetching a token with aws-cli.

If you've setup your kubeconfig to use aws-iam-authenticator, then ensure you've installed the AWS IAM Authenticator on the Jenkins master and that the jenkins user is able to use the binary (path settings).

This should get your past the 'system:anonymous' error during connection testing.

And finally, if you're hitting an error along the lines of Unauthorized! Token may have expired! Please log-in again. Unauthorized., then you need to ensure two things:

1. Ensure the plugin version for kubernetes-plugin is atleast 1.23.2. This will require a Jenkins version of atleast 2.190.1. This version is required for the step that follows.
2. This java setting is set in Jenkins Java options as mentioned here: JAVA_ARGS="-Dorg.csanchez.jenkins.plugins.kubernetes.clients.cacheExpiration=60". The default for this is 24 hours, whereas EKS expires tokens every 15 minutes. This setting for 60 clears the token every 60 seconds. `