How to Add a New Transform¶

TL;DR¶

Adding a transform is two steps: register a function in utils.py (or in an integration’s hook module), then reference it by name in config.yaml. There is no registry file to edit, no enum to update, and no import to add anywhere else.

Prerequisites¶

A working OAA-runner development environment (uv sync inside src/)
Familiarity with Python decorators
An understanding of what you want the transform to do

Step 1: Write and register the function¶

All transforms live in src/oaa/utils.py. Two decorators handle registration automatically.

Simple transform (no options)¶

Use @transformer_simple when the transform takes a value and returns a converted value with no configuration. The existing lowercase transform is the canonical example:

@transformer_simple
def lowercase(value, *args, **kwargs):
    return value.lower()

The function name (lowercase) becomes the YAML name: key automatically.

Factory transform (uses options from YAML)¶

Use @transformer(has_inner=True) when the transform needs configuration supplied via options: in config.yaml. The decorator calls the outer function once at setup time to build a configured inner callable that is then applied to every row. The existing truncate transform shows this pattern:

@transformer(has_inner=True)
def truncate(options=None, **kwargs):
    options = options or {}
    max_len = int(options.get('max_length', 255))

    def inner(value, *args, **kwargs):
        if isinstance(value, str):
            return value[:max_len]
        return value

    return inner

The options dict is populated from the options: block in config.yaml. Defaults should always be handled inside the outer function so the transform works without options too.

Mode-add transform (creates a new column)¶

Use @transformer(mode="add", has_inner=True) when the transform synthesises a new field rather than converting an existing one. The inner function receives the full row record rather than a single value. The jinja_render transform uses this pattern to build a new field from a Jinja2 template.

Step 2: Use it in config.yaml¶

The function name is the value of the name: key. No other changes are required — the DynamicTranformOptions_Enum in models.py is rebuilt from utils.FUNCS.keys() at import time, so the new name is automatically valid in field mapping validation.

Single transform¶

From docs/reference/config-examples/api-source.yaml:

field_mapping:
  department:
    destination_field: department_id
    transforms:
      - name: lowercase

Chaining transforms¶

Transforms run in order. From src/tests/configs/config2-multiple-transforms-one-field.yaml:

field_mapping:
  name:
    destination_field: name
    transforms:
      - name: jinja_render
        options:
          template: "{{row.firstName}} {{row.lastName}}"
      - name: lowercase
        options: {}

Transform with options and chaining¶

From src/tests/configs/config-v2-multi-datasource-csv.yaml:

field_mapping:
  IS_ADMIN:
    destination_field: is_admin
    custom_type: BOOLEAN
    transforms:
      - name: filter_by_field
        options:
          options: 'true'
      - name: validate_boolean

Factory transform with options¶

field_mapping:
  job_title:
    destination_field: job_title
    transforms:
      - name: truncate
        options:
          max_length: 24

Step 3: Add tests¶

All transform unit tests live in src/tests/test_transforms.py. Add a group of plain def test_*() functions — no test classes. Cover at minimum:

The normal/happy-path case
Edge cases (empty string, None, non-string input)
Each distinct options variant

Run the tests:

cd src
uv run python -m pytest tests/test_transforms.py -v

Integration-local transforms¶

If a transform is specific to one integration you do not need to add it to utils.py. Import the decorators directly into the integration’s hook module and register the function there:

from oaa.utils import transformer_simple, transformer

@transformer_simple
def my_local_transform(value, *args, **kwargs):
    # integration-specific logic
    return value.replace('_', '-')

Because the transformers registry is module-level, any import of oaa.utils that causes utils.py to run will also execute the hook module’s decorator, registering the name. Ensure the hook module is imported before the transform pipeline runs.

If the same transform turns out to be needed by multiple integrations, move it to utils.py and add it to what-builtin-transformers-are-available.md.