0

I want to run Apache Airflow using Docker on Windows to experiment with running tasks.

I'm new to Docker and have so far (1) initiated an Airflow instance that is running on Docker and (2) have created a DAG script locally.

What I want to do is to move this DAG script into a "dags" folder located somewhere in the container - I read that the directory is specified in airflow.cfg, but haven't been able to read it with either nano/vim.

(1) Do I have to create the 'dags' folder myself? (2) How would one copy over local scripts to the dags folder?

1 Answer 1

0

Found my answer. It's considered "hacky", may not be safe and may cause versioning issues to "directly" copy scripts into a Docker container.

Sharing a directory as a "volume" in Docker is the way to go:

docker run -d -p 8080:8080 -v /path/to/dags/on/your/local/machine/:/usr/local/airflow/dags  puckel/docker-airflow webserver

Source: Getting Started with Airflow using Docker

1
  • The puckel image is using a rather old version of airflow, for a slightly newer one use ednarb29/docker-airflow:2.2.3. Otherwise, if your machine can take it, there is the official docker-compose way: airflow.apache.org/docs/apache-airflow/stable/howto/… (and yes you have to create the dags folder next to the docker-compose file) Dec 3, 2022 at 23:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.