Dagu vs Airflow: setup and workflow comparison

Dagu is an open source, self-hosted workflow engine for existing scripts and containers. The video compares its single Go binary and file storage with the infrastructure the presenter associates with Apache Airflow. It describes Linux and Raspberry Pi use, including air-gapped operation without external services.

The setup walkthrough covers an installer script or Docker with a volume mount, followed by a web UI on port 8080. YAML files define steps, dependencies and cron schedules. Each step can have a retry policy, timeout and preconditions. The interface displays the dependency graph, logs and persistent run history, and lets a person approve a paused workflow.

Supported tasks discussed include shell commands, Docker containers, Kubernetes jobs, SSH tasks, HTTP requests, SQL queries and S3 operations. The presenter also suggests AI agent orchestration with LLM calls and approval steps, but does not demonstrate a local model setup. Workflow definitions, JSON run history, text logs and queue state reside on the filesystem.

The presenter recommends Dagu for tens to a few thousand workflow runs per day when tasks mostly invoke existing scripts or containers. That is a suggested fit, not a benchmark. For granular multi-tenancy, dynamically generated DAGs with millions of branches, or teams dependent on Airflow's Python ecosystem, the video recommends staying with Airflow.