Conductor (software) explained

Conductor
Developer:Netflix Team (Originally)Orkes & Open Source contributors (Current)
Latest Release Version:3.15.0
Programming Language:Java, JavaScript (UI), Python, Go
Platform:Linux
License:Apache License 2.0

Conductor is a free and open-source microservice orchestration software platform originally developed by Netflix.[1] [2]

Conductor was developed by Netflix to solve the problems of orchestrating microservices and business processes at scale in a cloud native environment.[3] It was released under the Apache License 2.0 and has been adopted by companies looking to orchestrate their processes at scale in a cloud native environment.[4] [5]

Conductor belongs to a set of software products that allows developers to build resilient, high-scale, cloud-native stateful applications using stateless primitives.

Architecture

Conductor server is written in Java with APIs exposed over HTTP and gRPC interfaces making it possible to do language agnostic development. A set of client libraries are made available by Netflix and community in Java, Python and Go.

Conductor uses a lightweight JSON based schema with rich programming language constructs such as fork/join, switch case, loops and exception handling to define the flows.

At the heart of Conductor is a queuing system that is used to schedule tasks and manage the process flows.  Conductor leverages a pluggable model allowing different implementations of the queuing system.  Open source version uses Dyno-Queues developed at Netflix for queuing as default implementation.[6]

The workflows are defined as the orchestration among the tasks which can be a system level construct such as fork, join, switch, loop, an external HTTP endpoint implementing business logic or a task worker running outside of Conductor servers and listening for work to be scheduled by the server. The workers communicate with the server using pre-defined APIs over HTTP or gRPC. Conductor provides lightweight libraries to manage worker states in Java, Python and Go and additional languages can be used to implement logic using provided APIs.

Conductor uses pluggable architecture model allowing for different databases to store its states. The current version has support for Redis (Stand-alone, Sentinel, Cluster and Dynomite), Postgres, Mysql, Cassandra and uses Elasticsearch as indexing mechanism.

The UI is written in ReactJS and provides ability to search, visualize and manage the workflow states.

Cloud Hosted Services

Orkes - was founded by the creators of Netflix Conductor and provides cloud managed offering of Netflix Conductor in various clouds and in data center.[7]

Notes and References

  1. Web site: Netflix Conductor, an Orchestration Engine for Microservices. 2021-09-22. InfoQ. en.
  2. Web site: Conductor, the newest tool from Netflix for orchestration of…. 2021-09-22. en.paradigmadigital.com. en.
  3. Web site: Blog. Netflix Technology. 2016-12-12. Netflix Conductor: A microservices orchestrator.
  4. Web site: 2016-12-13. Netflix open-sources Conductor, a microservices orchestration engine. 2021-09-22. JAXenter. en-US.
  5. Web site: Krill . Paul . Netflix open-sources its microservices orchestrator . InfoWorld . 23 September 2021 . en . 16 December 2016.
  6. Web site: Blog. Netflix Technology. 2017-04-19. Distributed delay queues based on Dynomite. 2021-09-19. Medium. en.
  7. Web site: Lunden . Ingrid . 2022-02-28 . Orkes, founded by the creators of Netflix's open source Conductor workflow orchestration tool, comes out of stealth with $9.3M . 2023-03-09 . TechCrunch . en-US.