23

I am testing out deploying my Django application into AWS's Fargate Service.

Everything seems to run, but I am getting Health Check errors as the Application Load Balancer is sending requests to my Django application using the Local Ip of the host. This give me an Allowed Host error in the logs.

Invalid HTTP_HOST header: '172.31.86.159:8000'. You may need to add '172.31.86.159' to ALLOWED_HOSTS

I have tried getting the Local ip at task start up time and appending it to my ALLOWED_HOSTS, but this fails under Fargate:

import requests

EC2_PRIVATE_IP  =   None
try:
    EC2_PRIVATE_IP  =   requests.get('http://169.254.169.254/latest/meta-data/local-ipv4', timeout = 0.01).text
except requests.exceptions.RequestException:
    pass

if EC2_PRIVATE_IP:
    ALLOWED_HOSTS.append(EC2_PRIVATE_IP)

Is there a way to get the ENI IP Address so I can append it to ALLOWED_HOSTS?

3 Answers 3

13

In fargate, there is an environment variable injected by the AWS container agent:${ECS_CONTAINER_METADATA_URI}

This contains the URL to the metadata endpoint, so now you can do

curl ${ECS_CONTAINER_METADATA_URI}

The output looks something like

{  
   "DockerId":"redact",
   "Name":"redact",
   "DockerName":"ecs-redact",
   "Image":"redact",
   "ImageID":"redact",
   "Labels":{  },
   "DesiredStatus":"RUNNING",
   "KnownStatus":"RUNNING",
   "Limits":{  },
   "CreatedAt":"2019-04-16T22:39:57.040286277Z",
   "StartedAt":"2019-04-16T22:39:57.29386087Z",
   "Type":"NORMAL",
   "Networks":[  
      {  
         "NetworkMode":"awsvpc",
         "IPv4Addresses":[  
            "172.30.1.115"
         ]
      }
   ]
}

Under the key Networks you'll find IPv4Address

Putting this into python, you get

METADATA_URI = os.environ['ECS_CONTAINER_METADATA_URI']
container_metadata = requests.get(METADATA_URI).json()
ALLOWED_HOSTS.append(container_metadata['Networks'][0]['IPv4Addresses'][0])

An alternative solution to this is to create a middleware that bypasses the ALLOWED_HOSTS check just for your healthcheck endpoint, eg

from django.http import HttpResponse
from django.utils.deprecation import MiddlewareMixin

class HealthEndpointMiddleware(MiddlewareMixin):
    def process_request(self, request):
        if request.META["PATH_INFO"] == "/health/":
            return HttpResponse("OK")
3
  • I have updated the answer to include the env var you listed. Thanks! Oct 30, 2019 at 15:03
  • This worked for me, because the response did not have a 'Containers' section like the accepted answer. I'm not sure whether this is because I'm only running one container (I'd certainly hope the format would be consistent regardless!) or because I'm running a newer version of the platform (1.3.0).
    – erik258
    Jan 23, 2020 at 2:11
  • 2
    For platform 1.4.0, the variable is now ECS_CONTAINER_METADATA_URI_V4. Ref: docs.aws.amazon.com/AmazonECS/latest/userguide/… Jan 29, 2021 at 18:14
13

Now this works, and it lines up with the documentation, but I don't know if it's the BEST way or if there is a BETTER WAY.

My containers are running under the awsvpc network mode.

https://aws.amazon.com/blogs/compute/under-the-hood-task-networking-for-amazon-ecs/

...the ECS agent creates an additional "pause" container for each task before starting the containers in the task definition. It then sets up the network namespace of the pause container by executing the previously mentioned CNI plugins. It also starts the rest of the containers in the task so that they share their network stack of the pause container. (emphasis mine)

I assume the

so that they share their network stack of the pause container

Means we really just need the IPv4 Address of the pause container. In my non-exhaustive testing it appears this is always Container[0] in the ECS meta: http://169.254.170.2/v2/metadata

With those assumption in play this does work, though I don't know how wise it is to do:

import requests

EC2_PRIVATE_IP = None
METADATA_URI = os.environ.get('ECS_CONTAINER_METADATA_URI', 'http://169.254.170.2/v2/metadata')

try:
    resp = requests.get(METADATA_URI)
    data = resp.json()
    # print(data)

    container_meta = data['Containers'][0]
    EC2_PRIVATE_IP = container_meta['Networks'][0]['IPv4Addresses'][0]
except:
    # silently fail as we may not be in an ECS environment
    pass

if EC2_PRIVATE_IP:
    # Be sure your ALLOWED_HOSTS is a list NOT a tuple
    # or .append() will fail
    ALLOWED_HOSTS.append(EC2_PRIVATE_IP)

Of course, if we pass in the container name that we must set in the ECS task definition, we could do this too:

import os
import requests

EC2_PRIVATE_IP = None
METADATA_URI = os.environ.get('ECS_CONTAINER_METADATA_URI', 'http://169.254.170.2/v2/metadata')

try:
    resp = requests.get(METADATA_URI)
    data = resp.json()
    # print(data)

    container_name = os.environ.get('DOCKER_CONTAINER_NAME', None)
    search_results = [x for x in data['Containers'] if x['Name'] == container_name]

    if len(search_results) > 0:
        container_meta = search_results[0]
    else:
        # Fall back to the pause container
        container_meta = data['Containers'][0]

    EC2_PRIVATE_IP = container_meta['Networks'][0]['IPv4Addresses'][0]
except:
    # silently fail as we may not be in an ECS environment
    pass

if EC2_PRIVATE_IP:
    # Be sure your ALLOWED_HOSTS is a list NOT a tuple
    # or .append() will fail
    ALLOWED_HOSTS.append(EC2_PRIVATE_IP)

Either of these snippets of code would then in in the production settings for Django.

Is there a better way to do this that I am missing? Again, this is to allow the Application Load Balancer health checks. When using ECS (Fargate) the ALB sends the host header as the Local IP of the container.

4
  • 1
    We do something similar , and it's working , in the entrypoint.sh add: export machineIP=$(echo `curl -s http://169.254.170.2/v2/metadata` | python -c "import sys, json; print json.load(sys.stdin)['Containers'][0]['Networks'][0]['IPv4Addresses'][0]") This will put the ip into and environment variable.
    – clay
    Jul 8, 2019 at 12:39
  • My data looked different from this. I suspect the ECS metadata 1.13 may be different.
    – erik258
    Jan 23, 2020 at 2:01
  • Why use only the first IP address from container_meta['Networks'][0]['IPv4Addresses'][0] and not all like container_meta['Networks'][0]['IPv4Addresses']? Jan 29, 2021 at 18:16
  • Much easier way of the entrypoint-method: export X_AWS_FARGATE_HOST_IP=$(hostname --ip-address). There is one small caveat though, this variable will not be available through a new shell (e.g. running the execute command on the task). Feb 28, 2022 at 20:47
2

I solved this issue by doing this:

First i've installed this middleware that can handle CIDR masks on top of ALLOWED_HOSTS: https://github.com/mozmeao/django-allow-cidr

With this middleware i can use a env var like this:

ALLOWED_CIDR_NETS = ['192.168.1.0/24']

So you need to find out the subnets you had configured on your ECS Service Definition, for me it was: 10.3.112.0/24 and 10.3.111.0/24.

You add that to your ALLOWED_CIDR_NETS and you're good to go.

1
  • worked for me, thanks!
    – Craig P
    Aug 21 at 14:17

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.