Apache DolphinScheduler

What Apache DolphinScheduler is

Apache DolphinScheduler is a free, open-source platform for designing and coordinating data workflows. Users can build visual DAG workflows with drag-and-drop tools or define them through Python, YAML, or the Open API. The project lists more than 30 built-in task types, including Shell, SQL, Spark, Flink, Hive, Kubernetes, and HTTP. Workflows can depend on other projects or workflows, pause and resume, and run batch backfills in serial or parallel mode. Its multi-master and multi-worker design is decentralized, and the project says scheduling capacity grows with cluster size. Deployment options include standalone, cluster, and Kubernetes. Standalone is intended for quick evaluation, not production; it is recommended for fewer than 20 workflows, and its default in-memory H2 database is cleared when the service stops or restarts. Security documentation lists password, LDAP, Casdoor SSO, and OAuth2 authentication, as well as administrator-managed controls for resources and data sources. The security model assumes a secure, trustworthy network and does not address internal network cross-infiltration. The project provides documentation, quick starts, and a community mailing list.