Series
Here is how to deploy Airflow in ECS in different ways, from Terraform to docker compose native integration.
Previously in Deploying Airflow in ECS using S3 as DAG storage via Terraform, I described how to deploy all components in AWS ECS using a hybrid EC2/Fargate launch type and S3 as DAG storage. Now let's do the same, but with three main differences: ...
Previously on How to set up a containerised Airflow installation in AWS ECS using Redis as its queue orchestrator, I gave an overview of the infrastructure and Airflow components. Now let's deploy all that. This deployment will incur charges!! Base...
How to set up a containerised Airflow installation in AWS ECS using Redis as its queue orchestrator. A bit of background A few years ago I joined a Data team where we processed a lot of analytics information coming from online search engines. This ET...
