22 Aug 2022

Django on ECS

Overview

In this post, we’ll create a Django app and deploy it to AWS ECS using Terraform. We’ll be using AWS Cloud9 as our development environment.

Django is a Python web framework that makes it easy to develop web apps quickly. We’ll be building a Django container image and deploying it with AWS ECS, a fully managed container orchestration service.

(It’s important to note that some of these steps will incur charges in your AWS account!)

Initial Setup

Set up Cloud9

From the Cloud9 Console in AWS, create a new environment. Call it whatever you want. Select Ubuntu Server for the platform and accept all the other defaults.

WARNING: THIS STEP WILL INCUR A COST!

Configure Cloud9’s credentials

When your environment is ready, we’ll need to prepare it to be able to run Terraform. Start by turning off AWS managed temporary credentials:

With the environment open, in the AWS Cloud9 IDE, on the menu bar choose AWS Cloud9, Preferences.
On the Preferences tab, in the navigation pane, choose AWS Settings, Credentials.
Use AWS managed temporary credentials to turn AWS managed temporary credentials off.

Follow the documentation to:

Create and use an instance profile to manage temporary credentials: https://docs.aws.amazon.com/cloud9/latest/user-guide/credentials.html#credentials-temporary
Attach an instance profile to an instance with the Amazon EC2 console: https://docs.aws.amazon.com/cloud9/latest/user-guide/credentials.html#credentials-temporary-attach-console

Create a Django Docker image

Create a new Django project

From your Cloud9 environment, run the following commands to create a new Django project:

$ mkdir django-ecs-terraform && cd django-ecs-terraform
$ mkdir app && cd app
$ python3 -m venv env
$ source env/bin/activate

(env)$ pip install django==3.2.9
(env)$ django-admin startproject django_aws .
(env)$ python3 manage.py migrate

Allow connections from Cloud9

Because you’re running on Cloud9, you will need to edit settings.py to allow connections from your Cloud9 instance’s domain name.

First, get your instance’s domain name by selecting Preview -> Preview running application. This will open a new tab within your Cloud9 environment. Copy the full URL from that tab (removing the https:// from the start). Then open settings.py and enter that into these two spots:

ALLOWED_HOSTS = ['979497206653481da585ce395e0c0c8f.vfs.cloud9.ap-southeast-2.amazonaws.com']
X_FRAME_OPTIONS = 'ALLOW-FROM 979497206653481da585ce395e0c0c8f.vfs.cloud9.ap-southeast-2.amazonaws.com'

Then start the server:

(env)$ python3 manage.py runserver 127.0.0.1:8080

You should be able to click the Preview button and see Django’s default page.

Build a container image

Add a requirements.txt:

Django==3.2.9
gunicorn==20.1.0

Create a Dockerfile:

FROM python:3.10-slim-buster

# Open http port
EXPOSE 8000

ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV DEBIAN_FRONTEND noninteractive

# Install pip and gunicorn web server
RUN pip install --no-cache-dir --upgrade pip
RUN pip install gunicorn==20.1.0

# install psycopg2 dependencies
RUN apt-get update \
  && apt-get -y install gcc postgresql \
  && apt-get clean

# Install requirements.txt
COPY requirements.txt /
RUN pip install --no-cache-dir -r /requirements.txt

# Moving application files
WORKDIR /app
COPY . /app

Build the image and run a container:

docker build . -t django-aws-backend

docker run -p 8080:8080 django-aws-backend gunicorn -b 0.0.0.0:8080 django_aws.wsgi:application

Publish container image to ECR

Set environment variables, replacing with your own AWS account ID and region of choice:

export AWS_ACCOUNT_ID=123456789012
export AWS_REGION=ap-southeast-2
export ECR_ENDPOINT=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com

Go into the AWS Console and create a private ECR repo called django-app.

Then build and push your image:

docker build -t $ECR_ENDPOINT/django-app:latest .

aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $ECR_ENDPOINT
docker push $ECR_ENDPOINT/django-app:latest

Add a health check

Because our Django instances are going to be behind a load balancer, we need a way to advertise the health of the service.

Edit app/django_aws/middleware.py and add these lines:

from django.http import HttpResponse
from django.utils.deprecation import MiddlewareMixin


class HealthCheckMiddleware(MiddlewareMixin):
    def process_request(self, request):
        if request.META['PATH_INFO'] == '/health/':
            return HttpResponse('Success')

Add the health check class to settings.py:

MIDDLEWARE = [
    'django_aws.middleware.HealthCheckMiddleware',  # new
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

Build and push your image again:

docker build -t $ECR_ENDPOINT/django-app:latest .

docker push $ECR_ENDPOINT/django-app:latest

Create a custom endpoint

We’re going to add a custom URL to represent one of the endpoints in our API:

python3 manage.py startapp polls

Open the file polls/views.py and enter this code:

from django.http import HttpResponse


def index(request):
    return HttpResponse("Hello, world!")

Now map this view to a URL. Create a file called polls/urls.py, containing the following code:

from django.urls import path

from . import views

urlpatterns = [
    path('', views.index, name='index'),
]

Now point the root URLconf at the polls.urls module. Open django_aws/urls.py, add an import for django.urls.include and insert an include() in the urlpatterns list:

from django.contrib import admin
from django.urls import include, path

urlpatterns = [
    path('polls/', include('polls.urls')),
    path('admin/', admin.site.urls),
]

Again, build and push your image:

docker build -t $ECR_ENDPOINT/django-app:latest .

docker push $ECR_ENDPOINT/django-app:latest

Create your AWS environment using Terraform

Clone the repository

Clone the git repository here: https://github.com/aynsof/django-ecs-terraform

This will deploy:

VPC
- Subnets - public and private
- Route tables
- Internet Gateway
- Key pairs
- Security Groups
IAM
- Roles
- Policies
ECS
- Cluster
- Task Definition
- Service
EC2
- Launch Config
- Auto Scaling Group
- Load Balancer
- Listeners
- Target Groups
- Health Checks
RDS
- Databse instance

Configure Terraform

Download Terraform from https://www.terraform.io/ - the simplest way is to download the Linux binary, unzip it, and put the binary in your $PATH.

For example, using the current release at the time of writing this post:

wget https://releases.hashicorp.com/terraform/1.2.7/terraform_1.2.7_linux_amd64.zip

unzip terraform_1.2.7_linux_amd64.zip

sudo mv terraform /usr/bin

Update variables and deploy

Update variables.tf with your region of choice:

# core

variable "region" {
  description = "The AWS region to create resources in."
  default     = "ap-southeast-2"
}

And with the relevant Availability Zones (AZs) for your region. In this case, ap-southeast-2a and ap-southeast-2b:

variable "availability_zones" {
  description = "Availability zones"
  type        = list(string)
  default     = ["ap-southeast-2a", "ap-southeast-2b"]
}

From the django-ecs-terraform directory, initialise Terraform:

terraform init

Generate the Terraform execution plan. This will show you what Terraform is planning to create:

terraform plan

Apply the Terraform to create the environment in your AWS account.

WARNING: THIS STEP WILL INCUR A COST!

terraform apply -auto-approve

You’ll be prompted for your desired database password, and for the URL of your ECR image.

If you want to avoid being prompted in future, you can create a terraform.tfvars file and enter the details in there:

docker_image_url_django = "123456789012.dkr.ecr.ap-southeast-2.amazonaws.com/django-app:latest"
rds_password = "random_password_goes_here"

Just make sure that your .gitignore contains a reference to *.tfvars before you push your code to a public git repository. You don’t want your database password to be added to version control!

Viewing the output

Terraform will give you the URL of your load balancer. It will look something like http://production-alb-1234567890.ap-southeast-2.elb.amazonaws.com/.

Add a /polls/ to the end of the URL so it looks like this: http://production-alb-1234567890.ap-southeast-2.elb.amazonaws.com/polls/.

When you visit this URL, you should see a Hello world! displayed.

Congratulations! You’ve successfully deployed a Django container to an AWS ECS cluster!

Troubleshooting

Access to the ECS instances

If you need to gain shell access to the ECS instances, you can use AWS Systems Manager Session Manager. This will allow you to open up an SSH session from within your browser.

From the AWS Console, open up Systems Manager. Under Node Management, select Session Manager. Choose Start Session.

Select one of your ECS instances and choose Start Session.

From within your Session Manager session, run sudo docker ps. You will get output that looks something like this:

$ sudo docker ps
CONTAINER ID   IMAGE                                                                 COMMAND                  CREATED        STATUS        PORTS                            NAMES
414107c683b6   123456789012.dkr.ecr.ap-southeast-2.amazonaws.com/django-app:latest   "gunicorn -w 3 -b :8…"   24 hours ago   Up 24 hours   0.0.0.0:49153->8000/tcp, :::49153->8000/tcp   ecs-django-app-11-django-app-bcc4f0c0c4fca5c18301
434c0d1999f3   amazon/amazon-ecs-agent:latest                                        "/agent"                 24 hours ago   Up 24 hours                            ecs-agent

This gives you useful information about your environment. For example, you might want to check whether your instance is running the right container. You can find the port that the container is listening on - in this case 49153 - and try to curl that port:

$ curl localhost:49153/polls/
Hello world!

Access to the running containers

If you want to start a shell on a running container, copy the container ID of your running Django container from the output of sudo docker ps. In this case, that’s 414107c683b6.

Then start a shell in that container:

sudo docker exec -it 414107c683b6 /bin/bash

Your shell prompt should change to something like:

root@414107c683b6:/app#

You’re now running a shell within your container’s context. From here you could perhaps run ps -ef to see whether your container is running the right processes:

root@414107c683b6:/app# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 Aug23 ?        00:00:10 /usr/local/bin/python /usr/local/bin/gunicorn -w 3 -b :8000 django_aws.wsgi:application
root         7     1  0 Aug23 ?        00:00:13 /usr/local/bin/python /usr/local/bin/gunicorn -w 3 -b :8000 django_aws.wsgi:application
root         8     1  0 Aug23 ?        00:00:13 /usr/local/bin/python /usr/local/bin/gunicorn -w 3 -b :8000 django_aws.wsgi:application
root         9     1  0 Aug23 ?        00:00:13 /usr/local/bin/python /usr/local/bin/gunicorn -w 3 -b :8000 django_aws.wsgi:application
root        21     0  4 00:31 pts/0    00:00:00 /bin/bash
root        27    21  0 00:31 pts/0    00:00:00 ps -ef

Cloud9 errors

If you’re running through Cloud9, you might get this error: “InvalidClientTokenId: The security token included in the request is invalid.”

This means Cloud9 is trying to use its AWS managed temporary credentials, rather than its instance profile. To fix this, run through the steps in Configure Cloud9's credentials: confirm that AWS managed temporary credentials are turned off, that your Cloud9 EC2 instance has the appropriate IAM instance profile attached to it, and that the profile policy has the appropriate permissions.

Edit this on GitHub

jk's blog