System slow to respond to request. What to do?

Question

I have an application container behind an Nginx reverse proxy. When I access the application via https, the POST request returns 302 and then the GET returns 200. See monitoring via browser:

Request URL: https://myapp.com
Request Method: POST
Status Code: 302 Found
Remote Address: 192.168.10.20:443
Referrer Policy: no-referrer-when-downgrade

Request URL: https://myapp.com
Request Method: GET
Status Code: 200 OK
Remote Address: 192.168.10.20:443
Referrer Policy: no-referrer-when-downgrade

The system takes a long time to respond. When I have many simultaneous accesses, the POST and GET time increases proportionally until the container is in the "Exit" state. The problem happens when the Django application tries to send multiple password reset emails. Could there be a wrong conversion from HTTP to HTTPS or vice versa? Should I change or set any header settings? I ask these questions because I have one reverse proxy behind another. The first handles DNS names and the second, which is a container, handles Django static files. The second proxy passes the request to the application container that uses a mysql database.

Below are some basic settings:

Primary server - Reverse Proxy Server

myapp.conf:

server {
    listen              443 ssl http2;
    server_name         myapp.com;

    proxy_intercept_errors on;
    include /usr/share/nginx/html/nginx-errors-ptbr/nginx-errors.conf;

    # SSL
    ssl_certificate     /etc/nginx/ssl/myapp.crt;
    ssl_certificate_key /etc/nginx/ssl/myapp.key;


    # logging
    access_log          /var/log/nginx/id.access.log;
    error_log           /var/log/nginx/id.error.log warn;

    # reverse proxy
    location / {
        proxy_pass http://192.168.2.15:8080;
        include    proxy.conf;
    }

}

# HTTP redirect
server {
    listen      80;
    server_name myapp.com;
    return      301 https://myapp.com$request_uri;
}

proxy.conf:

proxy_http_version                 1.1;
proxy_cache_bypass                 $http_upgrade;

# Proxy headers
proxy_set_header Upgrade           $http_upgrade;
proxy_set_header Connection        $connection_upgrade;
proxy_set_header Host              $host;
proxy_set_header X-Real-IP         $remote_addr;
proxy_set_header Forwarded         $proxy_add_forwarded;
proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host  $host;
proxy_set_header X-Forwarded-Port  $server_port;

# Proxy timeouts
proxy_connect_timeout              60s;
proxy_send_timeout                 60s;
proxy_read_timeout                 60s;

Secondary server - Reverse Proxy Container

upstream django {
    server cont-django-01:8001;
}

server {

    listen 8080;

    location / {
        proxy_pass http://django;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
    }

    location /static/ {
        alias /var/www/app/static/;
    }

}

Django Container Log (cont-django-01):

File "/usr/local/lib/python3.9/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python3.9/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python3.9/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
  File "/usr/local/lib/python3.9/socket.py", line 844, in create_connection
    raise err
  File "/usr/local/lib/python3.9/socket.py", line 832, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

Answer 1

Could there be a wrong conversion from HTTP to HTTPS or vice versa?

VERY unlikely. Why didn't you test it?

Should I change or set any header settings?

Not unless your application is looking for different headers.

There are lots of words here but very little information. On the one hand, you have a widely used/respected piece of software, known for its excellent performance characteristics and on the other hand a mystery application, probably written in-house which might be sending emails. You told us all about the former but nothing about the latter not what is in between them.

You could have:

1. Checked the timing information in the nginx logs

2. amended the nginx to include lots of other timing information

3. Checked the timing information in the application server logs (if it has any).

4. amended the app server log timing

5. tested a direct connection to the application server, bypassing the reverse proxy

6. monitored CPU and memory usage while running your testing

7. point the reverse proxy at a webserver offering static content with known, good performance characteristics.

...and that's even before considering what might be going inside the code.

The tiny amount of information you did provide about the application server suggests it is connecting to an SMTP server - in which case you could also be applying steps 3,4 and 5 there too.