0

I installed airflow using Kubernetes and have login the airflow UI. It shows all dags, but they are not shown correctly.

1/ There is no on/off buttons on the left of Dag name, it just show empty checkbox.

2/ The "Recent Tasks" and "DAG Runs" columns look like they are trying to load something;

3/ If I click and therefore go to any of DAG, it looks like it tries to load something;

I tried both airflow 2.0.0 and 1.10.11 and they show the same so it is not because of version.

What is the problem of the airflow and how to fix that?

enter image description here

enter image description here

------- here I provide more information according to Ofek Hod's suggestion: 1/ run "kubectl logs <pod_id> webserver", after I login airflow web UI, I got many http 404 response. e.g. 404 when login

after I click any dag in airflow WebUI I got some other 404 response 404 after I click dag

2 Answers 2

2

Airflow interprets all your .py files in your dags folder first, I guess something goes wrong there.
As a rule-of-thumb, first access webserver and scheduler logs (for kub kubectl logs?), maybe you can find a hint there.

If not, try first to make a "clean" airflow instance without any of your dags code or related .py files- point the dags folder to an empty directory and see what happens (better if you turn on example dags configuration).

If that works, add your .py files from the original dag folder incrementally until you find problematic code.

If it's not working, probably the scheduler or webserver are messed up, please check the logs again with better attention.

2
  • Ofek Hod: Thanks! I run kubectl logs and see some 404 response which maybe related. do you know why it showed 404? is it because I need to set some path for the apache web server's page?
    – user389955
    Oct 18, 2020 at 17:16
  • Thanks Ofek Hod. I found the reason. the package I use to setup k8s-airflow has a step to install ./airflow/www/compile_assets.sh using npm but it missed the step to install npm. so I added "apt install -y npm" in the step and now I see airflow page correctly. Thanks
    – user389955
    Oct 18, 2020 at 18:51
0

find the answer myself:

The package I use to setup k8s-airflow has a step to run ./airflow/www/compile_assets.sh using npm but the package missed the step to install npm. so I added "apt install -y npm" in the step and now I see airflow page correctly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.