Skip to content

Runnable

Image title

Orchestrate your functions, notebooks, scripts anywhere!!

Runner icons created by Leremy - Flaticon

Transform any Python function into a portable, trackable pipeline in seconds.

Step 1: Install

pip install runnable

Optional Features

Install optional features as needed: 

pip install runnable[notebook]    # Jupyter notebook execution
pip install runnable[docker]     # Container execution
pip install runnable[k8s]        # Kubernetes job executors
pip install runnable[s3]         # S3 storage backend
pip install runnable[examples]   # Example dependencies

Step 2: Your Function (unchanged!)

# Your existing function - zero changes needed
def analyze_sales():
    total_revenue = 50000
    best_product = "widgets"
    return total_revenue, best_product

Step 3: Make It Runnable

# Add main function → Make it runnable everywhere
from runnable import PythonJob

def main():
    job = PythonJob(function=analyze_sales)
    job.execute()
    return job  # REQUIRED: Always return the job object

if __name__ == "__main__":
    main()

🎉 Success!

You just made your first function runnable and got:

  • ✅ Automatic tracking: execution logs, timestamps, results saved
  • ✅ Reproducible runs: full execution history and metadata
  • ✅ Environment portability: runs the same on laptop, containers, Kubernetes

Your code now runs anywhere without changes!

Want to See More?

🔧 Same Code, Different Parameters (2 minutes)

Change parameters without touching your code:

# Function accepts parameters
def forecast_growth(revenue, growth_rate):
    return revenue * (1 + growth_rate) ** 3

from runnable import PythonJob

def main():
    job = PythonJob(function=forecast_growth)
    job.execute()
    return job  # REQUIRED: Always return the job object

if __name__ == "__main__":
    main()

# Run different scenarios anywhere:
# Local: RUNNABLE_PRM_revenue=100000 RUNNABLE_PRM_growth_rate=0.05 python forecast.py
# Container: same command, same results
# Kubernetes: same command, same results

# ✨ Every run tracked with parameters - reproducible everywhere
See complete parameter example
examples/11-jobs/passing_parameters_python.py
"""
The below example shows how to set/get parameters in python
tasks of the pipeline.

The function, set_parameter, returns
    - JSON serializable types
    - pydantic models
    - pandas dataframe, any "object" type

pydantic models are implicitly handled by runnable
but "object" types should be marked as "pickled".

Use pickled even for python data types is advised for
reasonably large collections.

Run the below example as:
    python examples/03-parameters/passing_parameters_python.py

"""

from examples.common.functions import write_parameter
from runnable import PythonJob, metric, pickled


def main():
    job = PythonJob(
        function=write_parameter,
        returns=[
            pickled("df"),
            "integer",
            "floater",
            "stringer",
            "pydantic_param",
            metric("score"),
        ],
    )

    job.execute()

    return job


if __name__ == "__main__":
    main()

Try it: uv run examples/11-jobs/passing_parameters_python.py

Why bother? No more "what parameters gave us those good results?" - tracked automatically across all environments.

🔗 Chain Functions, No Glue Code (3 minutes)

Build workflows that run anywhere unchanged:

# Your existing functions
def load_customer_data():
    customers = {"count": 1500, "segments": ["premium", "standard"]}
    return customers

def analyze_segments(customer_data):  # Name matches = automatic connection
    analysis = {"premium_pct": 30, "growth_potential": "high"}
    return analysis

# What you used to write (glue code):
# customer_data = load_customer_data()
# analysis = analyze_segments(customer_data)

# What Runnable needs (same logic, no glue):
from runnable import Pipeline, PythonTask

def main():
    pipeline = Pipeline(steps=[
        PythonTask(function=load_customer_data, returns=["customer_data"]),
        PythonTask(function=analyze_segments, returns=["analysis"])
    ])
    pipeline.execute()
    return pipeline  # REQUIRED: Always return the pipeline object

if __name__ == "__main__":
    main()

# Same pipeline runs unchanged on:
# • Your laptop (development)
# • Docker containers (testing)
# • Kubernetes (production)

# ✨ Write once, run anywhere - zero deployment rewrites
See complete pipeline example
examples/02-sequential/traversal.py
"""
You can execute this pipeline by:

    python examples/02-sequential/traversal.py

A pipeline can have any "tasks" as part of it. In the
below example, we have a mix of stub, python, shell and notebook tasks.

As with simpler tasks, the stdout and stderr of each task are captured
and stored in the catalog.
"""

from examples.common.functions import hello
from runnable import NotebookTask, Pipeline, PythonTask, ShellTask, Stub


def main():
    stub_task = Stub(name="hello stub")  # [concept:stub-task]

    python_task = PythonTask(  # [concept:python-task]
        name="hello python", function=hello, overrides={"argo": "smaller"}
    )

    shell_task = ShellTask(  # [concept:shell-task]
        name="hello shell",
        command="echo 'Hello World!'",
    )

    notebook_task = NotebookTask(  # [concept:notebook-task]
        name="hello notebook",
        notebook="examples/common/simple_notebook.ipynb",
    )

    # The pipeline has a mix of tasks.
    # The order of execution follows the order of the tasks in the list.
    pipeline = Pipeline(  # [concept:pipeline]
        steps=[  # (2)
            stub_task,  # (1)
            python_task,
            shell_task,
            notebook_task,
        ]
    )

    pipeline.execute()  # [concept:execution]

    return pipeline


if __name__ == "__main__":
    main()

Try it: uv run examples/02-sequential/traversal.py

Why bother? No more "it works locally but breaks in production" - same code, guaranteed same behavior.

🚀 Mix Python + Notebooks (5 minutes)

Different tools, portable workflows:

# Python prepares data, notebook analyzes - works everywhere
def prepare_dataset():
    clean_data = {"sales": [100, 200, 300], "regions": ["north", "south"]}
    return clean_data

from runnable import Pipeline, PythonTask, NotebookTask

def main():
    pipeline = Pipeline(steps=[
        PythonTask(function=prepare_dataset, returns=["dataset"]),
        NotebookTask(notebook="deep_analysis.ipynb", returns=["insights"])
    ])
    pipeline.execute()
    return pipeline  # REQUIRED: Always return the pipeline object

if __name__ == "__main__":
    main()

# This exact pipeline runs unchanged on:
# • Local Jupyter setup
# • Containerized environments
# • Cloud Kubernetes clusters

# ✨ No more environment setup headaches or "works on my machine"
See complete mixed workflow
examples/02-sequential/traversal.py
"""
You can execute this pipeline by:

    python examples/02-sequential/traversal.py

A pipeline can have any "tasks" as part of it. In the
below example, we have a mix of stub, python, shell and notebook tasks.

As with simpler tasks, the stdout and stderr of each task are captured
and stored in the catalog.
"""

from examples.common.functions import hello
from runnable import NotebookTask, Pipeline, PythonTask, ShellTask, Stub


def main():
    stub_task = Stub(name="hello stub")  # [concept:stub-task]

    python_task = PythonTask(  # [concept:python-task]
        name="hello python", function=hello, overrides={"argo": "smaller"}
    )

    shell_task = ShellTask(  # [concept:shell-task]
        name="hello shell",
        command="echo 'Hello World!'",
    )

    notebook_task = NotebookTask(  # [concept:notebook-task]
        name="hello notebook",
        notebook="examples/common/simple_notebook.ipynb",
    )

    # The pipeline has a mix of tasks.
    # The order of execution follows the order of the tasks in the list.
    pipeline = Pipeline(  # [concept:pipeline]
        steps=[  # (2)
            stub_task,  # (1)
            python_task,
            shell_task,
            notebook_task,
        ]
    )

    pipeline.execute()  # [concept:execution]

    return pipeline


if __name__ == "__main__":
    main()

Try it: uv run examples/02-sequential/traversal.py

Why bother? Your entire data science workflow becomes truly portable - no environment-specific rewrites.

🔍 Complete Working Examples

All examples in this documentation are fully working code! Every code snippet comes from the examples/ directory with complete, tested implementations.

Repository Examples

📁 Browse All Examples

Complete, tested examples organized by topic:

  • examples/01-tasks/ - Basic task types (Python, notebooks, shell scripts)
  • examples/02-sequential/ - Multi-step workflows and conditional logic
  • examples/03-parameters/ - Configuration and parameter passing
  • examples/04-catalog/ - File storage and data management
  • examples/06-parallel/ - Parallel execution patterns
  • examples/07-map/ - Iterative processing over data
  • examples/11-jobs/ - Single job execution examples
  • examples/configs/ - Configuration files for different environments

📋 All examples include:

  • ✅ Complete Python code following the correct patterns
  • ✅ Configuration files for different execution environments
  • ✅ Instructions on how to run them with uv run
  • ✅ Tested in CI to ensure they always work

🚀 Quick Start: Pick any example and run it immediately: 

git clone https://github.com/AstraZeneca/runnable.git
cd runnable
uv run examples/01-tasks/python_tasks.py

What's Next?

You've seen how Runnable transforms your code for portability and tracking. Ready to go deeper?

🎯 Master the Concepts → Jobs vs Pipelines Learn when to use single jobs vs multi-step pipelines

📊 Handle Your Data → Task Types Work with returns, parameters, and different data types

👁️ Visualize Execution → Pipeline Visualization Interactive timelines showing execution flow and timing

⚡ See Real Examples → Browse Repository Examples All working examples with full code in the examples/ directory

🚀 Deploy Anywhere → Production Guide Scale from laptop to containers to Kubernetes

🔍 Compare Alternatives → Compare Tools See how Runnable compares to Kedro, Metaflow, and other orchestration tools

Why Choose Runnable?

  • Easy to adopt, its mostly your code

    Your application code remains as it is. Runnable exists outside of it.

    • No API's or decorators or any imposed structure.

    Getting started

  • 🏗 Bring your infrastructure

    runnable is not a platform. It works with your platforms.

    • runnable composes pipeline definitions suited to your infrastructure.
    • Extensible plugin architecture: Build custom executors, storage backends, and task types for any platform.

    Infrastructure

  • 📝 Reproducibility

    Runnable tracks key information to reproduce the execution. All this happens without any additional code.

    Run Log

  • 🔁 Retry failures

    Debug any failure in your local development environment.

    Advanced Patterns

  • 🔬 Testing

    Unit test your code and pipelines.

    • mock/patch the steps of the pipeline
    • test your functions as you normally do.

    Testing Guide

  • 💔 Move on

    Moving away from runnable is as simple as deleting relevant files.

    • Your application code remains as it is.