24

I'm trying to use airflow to define a specific workflow that I want to manually trigger from the command line.

I create the DAG and add a bunch of tasks.

dag = airflow.DAG(
    "DAG_NAME",
    start_date=datetime(2015, 1, 1),
    schedule_interval=None,
    default_args=args)

I then run in the terminal

airflow trigger_dag DAG_NAME

and nothing happens. The scheduler is running in another thread. Any direction is much appreciated. Thank You

3
  • can you see your dag when u make a airflow list_dags? Jun 28, 2016 at 1:53
  • 2
    Also make sure that the DAG itself is switched on in the UI.
    – ilana917
    Aug 23, 2016 at 12:11
  • 3
    By default, the DAGs are in the 'paused' state, you will have to unpause them before you can trigger a DAG. To unpause a DAG from the CLI, airflow unpause dag_id. To unpause a DAG from the Web UI, go to the Web UI and turn on the DAG. Apr 17, 2018 at 4:14

5 Answers 5

38

I just encountered the same issue.

Assuming you are able to see your dag in airflow list_dags or via the web server then:

Not only did I have to turn on the dag in the web UI, but I also had to ensure that airflow scheduler was running as a separate process.

Once I had the scheduler running I was able to successfully execute my dag using airflow trigger_dag <dag_id>

My dag configuration is not significantly different from yours. I also have schedule_interval=None

3
  • 4
    I recently encountered the same issue as the poster - switching the DAG on in the UI allowed me to trigger a manual run just fine. Thanks! Jul 17, 2017 at 18:54
  • I had a similar issue trying to run one of the stock examples with airflow trigger_dag. In that case the missing bit was that even though the DAGs were loaded from the example_dags module they also have to be visible to the scheduler in the DAGS_FOLDER or it won't check them.
    – Aa'Koshh
    Jul 26, 2017 at 16:10
  • @Aa'Koshh I know it's old, but just to clarify that with the dag examples, they don't have to be in the DAGS_FOLDER. You just need to turn on the toggle from UI of the dag you want to trigger and it will work. Jul 8, 2019 at 17:48
10

You may have disabled the workflow. To enable the workflow manually. Open up the airflow web server by

$ airflow webserver -p 8080

Go to http://localhost:8080 . You should see the list of all available dags with a toggle button on/off. By default everything is set to off. Search for your dag and toggle your workflow. Now try triggering the workflow from terminal. It should work now.

1
  • sitting like 5 hours searching why my scheduler is not working and then found the dag was off... thanks mate!
    – Alveona
    Jul 26, 2021 at 10:20
6

first make sure your database connection string on the airflow is working, weather it be on postgres, sqlite(by default) or any other database. Then run the command

 airflow initdb

This command should not be showing any connection errors

Secondly make sure your webserver is running on a separate thread

airflow webserver

Then run your schdeuler on a different thread

airflow scheduler

Finally trigger your dag on a different thread after the scheduler is running

airflow trigger_dag dag_id

Also make sure the dag name and task are present in the dag and task list

airflow list_dags
airflow list_tasks dag_id

And if the dag is switched off in your UI then toggle it on.

1

You should 'unpause' the drag you what to trigger. use airflow unpause xxx_drag and then airflow trigger_dag xxx_drag and it should work.

0

airflow trigger_dag -e <execution_date> <dag_id>

1
  • 2
    An explanation would make your answer much more helpful
    – user_na
    Jul 8, 2021 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.