Airflow setup with Docker-compose (provided by airflow)

Create your virtual environment and then activate it (then,install requirements if needed)-

1. Run this on your terminal inside project at base root :

This docker-compose.yaml uses the official apache/airflow images and sets up scheduler, webserver (API), workers, triggerer, DB (Postgres), Redis, and an airflow-init service used for initialization

2. Create host folders & .env :

3. Initialize database & create the first user : (But make sure you have Docker Desktop and its running on your system)

4. Start the entire stack : (After airflow-init succeeds, start everything)

or for logs in foreground:

Check containers running :

5. The web UI is available at http://localhost:8080 :

Anytime you want to run the docker container (if not running):

Stop your running containers and docker compose :

Stop + remove containers and volumes (DB/logs reset):

Stop + remove containers, volumes, and images (fresh start next time):

To stop a specific container from the running containers list :

1. check running containers -

2. stop a running container by its id or name -

Know more about AI/LLM/ML projects handling and setup end-to-end : AI/ML end-to-end project Readme.