I am trying to get a compare and contrast of various job processing services in AWS like AWS Batch, MWAA (Managed Workflow for Apache Airflow).
Both are used for processing jobs. Just trying to understand when we should go for Batch and when we should go for MWAA.
AWS MWAA uses Apache Airflow to create workflows and DAGS (Directed Acyclic Graphs) with Python to orchestrate complex, dependent tasks. AWS Batch is, as the name states, a batch processing service that utilizes docker containers. Batch allows you to manage instance types and container sizes to fine tune costs of workloads. Batch unlike MWAA does not utilize any orchestration service, so apart from the queue that it utilizes, there is no way to control the workflow of tasks.
If you have a job that requires complex workflows, AWS MWAA will reduce the complexity of managing those workflows. If you require large scale containerized processing for image analysis, data conversion, etc., AWS Batch will allow you to easily control your workloads to optimize speed and cost.
It should also be noted that AWS Step Functions is also a very efficient way to orchestrate various types of processing and application tasks. For simple tasks, there is very little overhead and management compared to AWS Batch or MWAA.
