9

I thought the macro prev_execution_date listed here would get me the execution date of the last DAG run, but looking at the source code it seems to only get the last date based on the DAG schedule.

prev_execution_date = task.dag.previous_schedule(self.execution_date)

Is there any way via macros to get the execution date of the DAG when it doesn't run on a schedule?

1

2 Answers 2

12

Yes, you can define your own custom macro for this as follows:

# custom macro function
def get_last_dag_run(dag):
    last_dag_run = dag.get_last_dagrun()
    if last_dag_run is None:
        return "no prev run"
    else:
        return last_dag_run.execution_date.strftime("%Y-%m-%d")

# add macro in user_defined_macros in dag definition
dag = DAG(dag_id="my_test_dag",
      schedule_interval='@daily',
      user_defined_macros={
          'last_dag_run_execution_date': get_last_dag_run
      }
)

# example of using it in practice
print_vals = BashOperator(
    task_id='print_vals',
    bash_command='echo {{ last_dag_run_execution_date(dag) }}',
    dag=dag
)

Note that the dag.get_last_run() is just one of the many functions available on the Dag object. Here's where I found it: https://github.com/apache/incubator-airflow/blob/v1-10-stable/airflow/models.py#L3396

You can also tweak the formatting of the string for the date format, and what you want output if there is no previous run.

1
  • 2
    dag.get_last_dagrun(include_externally_triggered=True) is for externally triggering dags
    – Hasitha
    Feb 7, 2020 at 7:34
0

You can make your own user custom macro function, use airflow model to search meta-database.

def get_last_dag_run(dag_id):
  //TODO search DB
  return xxx

dag = DAG(
    'example',
    schedule_interval='0 1 * * *',
    user_defined_macros={
        'last_dag_run_execution_date': get_last_dag_run,
    }
)

Then use the KEY in your template.

1
  • 2
    This answer seems to only answer one half, omitting the DB search.
    – tobi6
    Aug 20, 2018 at 7:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.