Skip to main content

Questions tagged [apache-airflow]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
139 views

How widespread, used is CWL, common workflow language?

Trying to get an overview of different languages and software for workflows and data pipelines--and thus 'data engineering'--I stumbled upon Common Workflow Language. I thought, well, how common is ...
Johan's user avatar
  • 103
5 votes
1 answer
3k views

When to use Apache Airflow (MWAA) in AWS instead of AWS Batch?

I am trying to get a compare and contrast of various job processing services in AWS like AWS Batch, MWAA (Managed Workflow for Apache Airflow). Both are used for processing jobs. Just trying to ...
Biju's user avatar
  • 255
0 votes
1 answer
134 views

Apache Airflow federation - observe remote dag in full detail?

We have numerous airflow instances running in different organizations/teams around the company. While we have built a highly scaleable centralized airflow service, we do have cases where we need to ...
John Humphreys's user avatar