Netflix Reworks Conductor for 420 Million Monthly Workflow Executions and 10X Larger Workflows

Leela Kumili — Fri, 11 Sep 2026 14:17:00 GMT

Netflix has reworked its Conductor workflow orchestration engine to handle larger workloads, increasing supported workflow size from about 2,500 to 30,000 tasks and reducing p99 workflow evaluation latency by about 40%. Conductor 4.0 separates workflow metadata from task data, moves evaluation to asynchronous processing, and introduces dynamic worker allocation and concurrency controls.

By Leela Kumili

Netflix Moves toward Open Source Flink Autoscaler for 30,000+ Streaming Jobs

Leela Kumili — Mon, 07 Sep 2026 14:06:00 GMT

Netflix is moving toward the open-source Apache Flink Autoscaler for more than 30,000 streaming jobs across multiple AWS regions. The operator-level approach addresses limitations of Netflix’s cluster level autoscaler for complex, stateful pipelines. Netflix reports a 58% reduction in annualized Flink compute expenditure for one team, saving approximately $1.1 million annually.

By Leela Kumili

InfoQ - Apache Kafka

Netflix Reworks Conductor for 420 Million Monthly Workflow Executions and 10X Larger Workflows

Netflix Moves toward Open Source Flink Autoscaler for 30,000+ Streaming Jobs