3

I have made a simple flask app to practice Pulumi.

It gets env variable set through Dockerfile and I intend to host it on AWS Fargate, and RDS Postgres as database.

Here is the Flask app:

import os

from flask import Flask, request
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate

app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = "postgresql://{}".format(
    os.environ.get("DATABASE_URL")
)
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db = SQLAlchemy(app)
migrate = Migrate(app, db)


class CarsModel(db.Model):
    __tablename__ = "cars"

    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String())
    model = db.Column(db.String())
    doors = db.Column(db.Integer())

    def __init__(self, name, model, doors):
        self.name = name
        self.model = model
        self.doors = doors

    def __repr__(self):
        return f"<Car {self.name}>"


@app.route("/")
def hello():
    return {"hello": "world"}


@app.route("/cars", methods=["POST", "GET"])
def handle_cars():
    if request.method == "POST":
        if request.is_json:
            data = request.get_json()
            new_car = CarsModel(
                name=data["name"], model=data["model"], doors=data["doors"]
            )

            db.session.add(new_car)
            db.session.commit()

            return {"message": f"car {new_car.name} has been created successfully."}
        else:
            return {"error": "The request payload is not in JSON format"}

    elif request.method == "GET":
        cars = CarsModel.query.all()
        results = [
            {"name": car.name, "model": car.model, "doors": car.doors} for car in cars
        ]

        return {"count": len(results), "cars": results, "message": "success"}


@app.route("/cars/<car_id>", methods=["GET", "PUT", "DELETE"])
def handle_car(car_id):
    car = CarsModel.query.get_or_404(car_id)

    if request.method == "GET":
        response = {"name": car.name, "model": car.model, "doors": car.doors}
        return {"message": "success", "car": response}

    elif request.method == "PUT":
        data = request.get_json()
        car.name = data["name"]
        car.model = data["model"]
        car.doors = data["doors"]

        db.session.add(car)
        db.session.commit()

        return {"message": f"car {car.name} successfully updated"}

    elif request.method == "DELETE":
        db.session.delete(car)
        db.session.commit()

        return {"message": f"Car {car.name} successfully deleted."}


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8000)

Here is the Dockerfile:

# Use an official Python runtime as a parent image
FROM python:3.8

# Set the working directory to /app
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY . /app

# Install any needed packages specified in requirements.txt
RUN pip install --trusted-host pypi.python.org -r requirements.txt

ENV FLASK_APP main.py
ENV DATABASE_URL localhost
RUN flask db init
RUN flask db migrate
RUN flask db upgrade
# Make port 80 available to the world outside this container
EXPOSE 8000

# Run app.py when the container launches
CMD ["python", "main.py"]

Here is the index.ts file for Pulumi:

import * as awsx from "@pulumi/awsx";
import * as aws from "@pulumi/aws";
import * as pulumi from "@pulumi/pulumi";

const vpc = new awsx.ec2.Vpc("custom");
// Step 1: Create an ECS Fargate cluster.
const cluster = new awsx.ecs.Cluster("first_cluster", { vpc });

const securityGroupIds = cluster.securityGroups.map(g => g.id);

const dbSubnets = new aws.rds.SubnetGroup("dbsubnets", {
    subnetIds: vpc.publicSubnetIds,
});

const db = new aws.rds.Instance("postgresdb", {
    engine: "postgres",

    instanceClass: "db.t2.micro",
    allocatedStorage: 20,

    dbSubnetGroupName: dbSubnets.id,
    vpcSecurityGroupIds: securityGroupIds,

    name: "dummy",
    username: "dummy",
    password: "123456789",
    publiclyAccessible: true,
    skipFinalSnapshot: true,
});
const hosts = pulumi.all([db.endpoint.apply(e => e)]);
const environment = hosts.apply(([postgresHost]) => [
    { name: "DATABASE_URL", value: postgresHost },
]);

// Step 2: Define the Networking for our service.
const alb = new awsx.elasticloadbalancingv2.ApplicationLoadBalancer(
    "net-lb", { external: true, securityGroups: cluster.securityGroups, vpc });
const atg = alb.createTargetGroup(
    "app-tg", { port: 8000, deregistrationDelay: 0 });
const web = atg.createListener("web", { port: 80, external: true });

// Step 3: Build and publish a Docker image to a private ECR registry.
const img = awsx.ecs.Image.fromPath("app-img", "./app");

// Step 4: Create a Fargate service task that can scale out.
const appService = new awsx.ecs.FargateService("app-svc", {
    cluster,
    taskDefinitionArgs: {
        container: {
            image: img,
            cpu: 102 /*10% of 1024*/,
            memory: 50 /*MB*/,
            portMappings: [web],
            environment: environment,
        },
    },
    desiredCount: 5,
}, { dependsOn: [db] });

// Step 5: Export the Internet address for the service.
export const url = web.endpoint.hostname;

Now, when I do pulumi up, I get this:

 sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
        Is the server running on host "localhost" (127.0.0.1) and accepting
        TCP/IP connections on port 5432?
    could not connect to server: Cannot assign requested address
        Is the server running on host "localhost" (::1) and accepting
        TCP/IP connections on port 5432?

    (Background on this error at: http://sqlalche.me/e/e3q8)

        at /Users/myuser/projects/practice/pulumi/simple_flask_app/node_modules/@pulumi/docker.ts:546:15
        at Generator.next (<anonymous>)
        at fulfilled (/Users/myuser/projects/practice/pulumi/simple_flask_app/node_modules/@pulumi/docker/docker.js:18:58)
        at processTicksAndRejections (internal/process/task_queues.js:97:5)

    error: The command '/bin/sh -c flask db migrate' returned a non-zero code: 1

Now, I know that its because its trying to connect to localhost as that is the default, but how to pass in the host name of db resource?

Thanks


UPDATE 1: Tried removing ENV DATABASE_URL localhost


After removing ENV DATABASE_URL localhost:

File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 652, in __connect
    connection = pool._invoke_creator(self)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 490, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not translate host name "None" to address: Name or service not known
(Background on this error at: http://sqlalche.me/e/e3q8)

    at /Users/myuser/projects/practice/pulumi/simple_flask_app/node_modules/@pulumi/docker.ts:546:15
    at Generator.next (<anonymous>)
    at fulfilled (/Users/myuser/projects/practice/pulumi/simple_flask_app/node_modules/@pulumi/docker/docker.js:18:58)
5
  • I may be reading it wrong, but I'd say you just need to remove ENV DATABASE_URL localhost from the Dockerfile so that the value from FargetService takes over Apr 13, 2020 at 19:07
  • @MikhailShilkov I tried that too, but it doesnt work, as pulumi up command builds the image. I have updated my question accordingly.
    – Maverick
    Apr 13, 2020 at 19:11
  • This example looks very similar to yours github.com/pulumi/examples/tree/… Apr 13, 2020 at 19:14
  • I took example from pulumi.com/docs/tutorials/aws/aws-ts-airflow but it doesnt work.
    – Maverick
    Apr 13, 2020 at 19:16
  • @MikhailShilkov the example you sent, has different needs, in the Dockerfile I have am running migration commands, that is where the problem comes in. So, it wont matter if the env variable is in Dockefile ot not.
    – Maverick
    Apr 14, 2020 at 7:07

1 Answer 1

-1

I’d consider this bad practice to run the migrations during the docker build. What happens if the build fails afterwards? How can you control which changes are applied to which environment? I think there are better solutions to this problem.

Those migrations could also be applied when the container boots up in fargate by e.g. putting those commands into an entrypoint script or executing the migration in the process startup (basically in your main.py) like described here: https://flask-migrate.readthedocs.io/en/latest/#command-reference

Another reason for not doing it during the pulumi up is that this would also require a firewall rule allowing your local machine to access the database (might be already “solved“ with your publiclyAccessible setting though).

If you still want to keep this action in the build, you need a different way of providing the database url to step 3. The env is only used during step 4 (setting up fargate). For step 3 you could leverage build args (https://docs.docker.com/engine/reference/builder/#arg) and pass them via pulumi like so https://www.pulumi.com/docs/reference/pkg/docker/image/#dockerbuild

Keep in mind that this adds some security issues because you open up the database to the public which wouldn’t be necessary otherwise. So I’d definitely go with a different approach as described above.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.