32

I was learning apache airflow and found that there is an operator called DummyOperator. I googled about its use case, but couldn't find anything that I can understand. Can anyone here please discuss its use case?

1
  • 5
    Why is this question being downvoted? I came here because I asked myself the same thing.
    – Alechan
    Nov 26, 2019 at 19:40

3 Answers 3

20

Operator that does literally nothing. It can be used to group tasks in a DAG.

https://airflow.apache.org/_api/airflow/operators/dummy_operator/index.html

as far as I know, at least to two case:

  • test purpose. in dags, the dummy operation just between upstream and downstream, later, you can replace the true operator.
  • Workflow purpose: BranchPythonOperator work with DummyOperator. If you want to skip some tasks, keep in mind that you can’t have an empty path, if so make a dummy task. https://airflow.apache.org/concepts.html#workflows
4
  • 14
    adding another use-case here: if you have a set of 'd' downstream tasks that must all execute after 'u' upstream tasks, then rather than creating a 'mesh' of 'u x d' connections in between, you can introduce a dummyoperator in between these two sets of tasks and get away with 'u + d' connections. This certainly provides an aesthetic improvement in rendering of your DAG (looks simpler, lighter to render), but can also potentially decrease the scheduling overhead (and scheduler latency) since much lesser number of conditions have to be evaluated to check scheduling-ready state Aug 14, 2019 at 4:30
  • @Yong Wang how can I keep some latency between two tasks ...i.e. after first task successful wait 1 hr and start second task.......how to check how my first job executed with shellscript is success or not ?
    – BdEngineer
    May 27, 2020 at 5:21
  • 1
    @BdEngineer Thanks for question. As far as I think the most convenient mode is to create a bashoperator which sleet 1hour'. And add it between the upstream task and downstream task. In this way, the original downstream task will be delay 1 hour after original upsteam task successed.. Hope it help you.
    – Yong Wang
    May 27, 2020 at 6:58
  • @y2k-shubham vote for the sense of aesthetic. Clean and elegant way.
    – Yong Wang
    May 27, 2020 at 7:01
6

dummy_operator is used in BranchPythonOperator where we decide next task based on some condition.

For example:

                  -> task C->task D

 task A -> task B                      -> task F
                   ->  task E(Dummy)

So let's suppose we have some condition in task B which decides whether to follow [task C->task D] or task E(Dummy) to reach task F.

Since we cannot leave else condition empty we have to put dummy operator which does nothing just skip or bypass.

1
  • 3
    Why can't the condition create a dependency directly from task B to task F? Mar 17, 2021 at 0:28
2

Another use case: I've implemented a framework that returns an Operator. In most cases this is a PostgresOperator but under some user-specified configuration there's no SQL to run but the caller still expects an Operator so I return a DummyOperator rather than a PostgresOperator with trivial SQL like "select 1;".

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.