Skip to content

local

All the steps of the pipeline are executed in the local compute environment in the same shell as it was triggered.

  • Provides the most comfortable environment for experimentation and development.
  • The scalability is constrained by the local compute environment.
  • Not possible to provide specialized compute environments for different steps of the pipeline.
  • All the steps within parallel or map nodes are executed sequentially.

Configuration

In the mode of local execution, we run everything on the local computer.

This has some serious implications on the amount of time it would take to complete the run. Also ensure that the local compute is good enough for the compute to happen of all the steps.

Example config:

pipeline-executor:
  type: local

All the examples in the concepts section are executed using local executors.