I work on an AWS Fargate service which has a health check configured as well as an autoscaling policy set.
The application needs about 30 - 45 seconds to start.
The application does receive traffic spikes now and then. The autoscaling policy will kick in when CPU usage goes above 60% for the last 3 minutes. The 1 minute resolution is the most fine grained resolution AWS provides for built-in metrics.
How can I configure the health checks and autoscaling policy so that autoscaling gets the chance to scale the service before health checks restarts the service?
Autoscaling needs at least 3 data points, and with 1 minute resolution, that's 3 minutes. The additional tasks started by autoscaling needs to start up, and they also need to start receive traffic before they can offload the existing tasks. However, I believe a task must pass "healthy threshold count" before it starts to receive traffic. So, this all adds up.
Is there a better way to coordinate autoscaling and health checks?
