Overview

Opt out

Pipelines need not use the secrets if the preferred tools of choice is not implemented in runnable. The default configuration of do-nothing is no-op by design. We kindly request to raise a feature request to make us aware of the eco-system.

Most complex pipelines require secrets to hold sensitive information during task execution. They could be database credentials, API keys or any information that need to present at the run-time but invisible at all other times.

runnable provides a clean API to access secrets and independent of the actual secret provider, the interface remains the same.

A typical example would be a task requiring the database connection string to connect to a database.

Using the secrets API

class CustomObject:

    @property
    def connection_object(self):
        from runnable import get_secret
        connection_string = get_secret("connection_string")
        # Do something with the secrets

Please refer to configurations for available implementations.

Example

dotenv formatExample configurationPipeline in python

The dotenv format for providing secrets. Ideally, this file should not be part of the version control but present during development phase.

The file is assumed to be present in examples/secrets.env for this example.

1. Shell scripts style are supported.
2. Key value based format is also supported.

Configuration to use the dotenv format file.

secrets:
  type: dotenv # (1)
  config:
    location: examples/secrets.env # (2)

1. Use dotenv secrets manager.
2. Location of the dotenv file, defaults to .env in project root.

1. The key of the secret that you want to retrieve.