Failing to authenticate with GHCR after kubernetes upgrade to 1.24

Question

I’ve upgraded my AWS EKS to 1.24, and since then authentication with github docker registry fails.

I’m running the following Ansible task in Jenkins:

- name: Login to github docker registry
  community.general.docker_login:
    registry: ghcr.io
    username: "{{ github_cred.user }}"
    password: "{{ github_cred.token }}"
    reauthorize: yes

And getting the following error:

19:04:41 redirecting (type: modules) community.general.docker_login to community.docker.docker_login
19:04:41 redirecting (type: modules) community.general.docker_login to community.docker.docker_login
19:04:42 fatal: [localhost]: FAILED! => changed=false 
19:04:42 msg: 'Error connecting: Error while fetching server API version: (''Connection aborted.'', ConnectionRefusedError(111, ''Connection refused''))'

I know that client.authentication.k8s.io/v1alpha1 is deprecated in 1.24, but not sure it’s relevant.

Does anyone know what could be the issue?

Answer 1

The error message you're encountering,

Error connecting: Error while fetching server API version: ("Connection aborted.", ConnectionRefusedError(111, "Connection refused"))

indicates a connection issue between your Ansible script running in Jenkins and the Docker registry. There are a few potential causes and solutions to explore:

1. Network Connectivity:

Check Firewall: Ensure no firewall rules are blocking communication between the Jenkins server and the GitHub container registry (ghcr.io).

DNS Resolution: Verify if DNS can properly resolve the hostname "ghcr.io" to an IP address. You can test this using nslookup ghcr.io in your terminal.

Network Reachability: Make sure there's proper network connectivity between Jenkins and the internet. You can ping a public website like "8.8.8.8" to test.

2. Authentication Issues:

Credentials: Double-check the values in your github_cred dictionary. Ensure the "user" is the correct username for accessing the GitHub container registry and the "token" is a valid Personal Access Token (PAT) with "read_repository" and "write_repository" scopes.

Secret Management: If you're storing the credentials in a Jenkins credential store or environment variable, confirm they are being injected into the Ansible task correctly.

3. Ansible Configuration:

Docker Module: While your code snippet shows community.general.docker_login, Ansible 2.9 and above use community.docker.docker_login. Update the module name as shown in the error message.

Docker Version: Using an older Docker client version might lead to compatibility issues. Consider upgrading the Docker client on your Jenkins server if possible.

4. Removed Feature (Unlikely):

Although you mentioned the deprecation of client.authentication.k8s.io/v1alpha1, it's highly unlikely to be directly causing your issue. This feature is related to Kubernetes authentication within the cluster, not Docker registry access.

Troubleshooting steps:

• Start by verifying network connectivity and resolving any firewall or DNS issues.
• Double-check your credentials and ensure they are accessible by the Ansible task running in Jenkins.
• Update the Ansible module name to community.docker.docker_login if using Ansible 2.9 or above.
• Consider upgrading the Docker client version on your Jenkins server if possible.

If the issue persists after trying these solutions, consider providing more details about your environment, including the specific Ansible version and any relevant security configurations in your EKS cluster. This additional information might help diagnose the issue further.