Skip to main content

Hashnode Implementations and Cloud

Implementations and Cloud

WWW

Command Palette

Search for a command to run...

Series

Airflow in ECS

Here is how to deploy Airflow in ECS in different ways, from Terraform to docker compose native integration.

Airflow in ECS with Redis - Part 3: docker compose
Previously in Deploying Airflow in ECS using S3 as DAG storage via Terraform, I described how to deploy all components in AWS ECS using a hybrid EC2/Fargate launch type and S3 as DAG storage. Now let's do the same, but with three main differences: ...
Mar 3, 20237 min read150
Airflow in ECS with Redis - Part 2: Hands On
Previously on How to set up a containerised Airflow installation in AWS ECS using Redis as its queue orchestrator, I gave an overview of the infrastructure and Airflow components. Now let's deploy all that. This deployment will incur charges!! Base...
Feb 24, 20237 min read328
Airflow in ECS with Redis - Part 1: Overview
How to set up a containerised Airflow installation in AWS ECS using Redis as its queue orchestrator. A bit of background A few years ago I joined a Data team where we processed a lot of analytics information coming from online search engines. This ET...
Jan 28, 20234 min read332

© 2026 Implementations and Cloud

Members
Archive
Privacy
Terms

Airflow in ECS