Do I need to do anything to manage many outgoing connections to an outside service from a kubernetes cluster?

Question

In my kubernetes cluster I'm running many small pods per node (about 30) and each pod creates a few TCP connections to a single service on the internet to send HTTP requests to. So each node on the cluster might have ~60 connections to the service at any point in time.

Do I need to do something to optimize this outgoing connection count? For example, can running a reverse proxy as a daemonset and connecting to the reverse proxy instead of directly connecting to the service on the internet help? Do I need to change some settings on the k8s node to allow that many outgoing connections? Or is the current setup fine as it is and doesn't need any optimization?

I am asking this because sometimes TCP connection from the pod to the service on the internet takes a very long time to establish, but a different app running outside the cluster doesn't have a problem connecting to the service.

Answer 1

In order to manage a large number of outbound connections to an external service from your K8s cluster, you may need to improve your setup. Each node (30) has a large number of small pods connected to a single external service via TCP connections, for a total of about 60 connections per node. You are finding that these pods take a long time to connect.

Potential causes of slow connections:

Numerous connections per node might put a burden on resources and delay the formation of connections. Each pod creating its own connections may lead to overhead and inefficiencies.

Yes, Recommended method is Reverse proxy:

Using a reverse proxy is the generally recommended approach for managing a large number of outgoing connections efficiently. It centralizes connection management and reduces overhead on individual pods. Refer to Radware article on Reverse proxy for more details.

• Install a reverse proxy on every node, such as HAproxy or NGINX, as a daemonset.
• Set the pods up so that they link to the internal reverse proxy rather than the external service.
• The connection pool to the remote service is managed by reverse proxy. Decreasing the total number of connections and increasing productivity.

Alternative method is Connection Pooling:

Consider using libraries within your pods that implement connection pooling for the specific service you are connecting to. These libraries can reuse existing connections reducing the need for frequent new connections. Refer to Michael Aboagye Stackoverflow blog on Connection pooling for more details.

If you are using connection pooling libraries ensure compatibility with your programming language and the service you are connecting to.

Monitor your cluster performance and connection times analyze metrics related to network traffic and resource usage to gauge effectiveness.