How to Use Generators to Create Groups and Roles from Field Values

Goal

Automatically create LocalGroup or LocalRole objects from unique field values in your source data — without writing a separate source or custom hook.

For example, if your users CSV contains a department column, a generator can read each unique department name and create a corresponding LocalGroup, then link each user to the group that matches their department.

When to Use Generators

Use a generator when:

  • You want to create groups or roles derived from data on another object (e.g. create a group per department, a role per job title).

  • The groups/roles don’t exist as a dedicated source in your data.

  • You want membership to be assigned automatically at the same time.

If your groups or roles already exist as a dedicated table or file, use a separate source with destination: LocalGroup or destination: LocalRole instead.

How Generators Work

Generators are declared inline under a sources.<name>.generators section. For each record in the source, the runner:

  1. Reads the value of source_field from the current record.

  2. Renders name_template and unique_id_template using that value.

  3. Creates a LocalGroup or LocalRole with the rendered name and unique ID — or retrieves the existing one if it was already created by a previous record.

  4. Optionally calls add_group() or add_role() on the primary source object (e.g. the LocalUser) to assign membership.

This means deduplication is automatic: 100 users with department = "Engineering" produce exactly one group.

Step-by-Step Example

1. Identify the source field

Find the column in your source data that contains the values you want to turn into groups or roles. In this example the source is a CSV of users:

login,email,department,job_title
alice@example.com,alice@example.com,Engineering,Backend Engineer
bob@example.com,bob@example.com,Marketing,Content Strategist
carol@example.com,carol@example.com,Engineering,Frontend Engineer

The columns department and job_title will each drive a generator.

2. Add a generators block to your source

version: v2
app:
  name: My App
  application_type: MyApp

sources:
  users:
    destination: LocalUser
    source_type: csv
    csv_filepath: data/users.csv
    field_mapping:
      login:
        destination_field: unique_id
      email:
        destination_field: email
      department:
        destination_field: department
      job_title:
        destination_field: job_title

    generators:
      department_groups:
        generator_type: local_group
        source_field: department
        name_template: "{{value}}"
        unique_id_template: "dept_{{value}}"
        add_membership: true
        skip_empty: true

      title_roles:
        generator_type: local_role
        source_field: job_title
        name_template: "{{value}}"
        unique_id_template: "role_{{value}}"
        add_membership: true
        skip_empty: true

3. Understand the config keys

Key

Required

Default

Description

generator_type

✅

—

local_group or local_role

source_field

✅

—

Field name to read values from

name_template

{{value}}

Jinja2 template for the object’s name

unique_id_template

{{value}}

Jinja2 template for the object’s unique_id

add_membership

true

Link the source record to the generated object

skip_empty

true

Skip records where source_field is blank or null

multi_value

false

Treat source_field as a Python list; iterate each element

create_if_missing

true

When false, only link to existing objects — never create

field_mapping

—

Optional: set extra properties on the generated object

4. Template variables

Both name_template and unique_id_template are Jinja2 templates. Two variables are available:

  • {{value}} — the value read from source_field for the current record.

  • {{row}} — the full source record (access other fields as {{row.department}}).

Examples:

# Simple: use the value directly
name_template: "{{value}}"

# Add a prefix
unique_id_template: "dept_{{value}}"

# Combine fields
name_template: "{{row.division}} / {{value}}"

5. Set additional properties using field_mapping (optional)

You can set extra properties on the generated object using an optional field_mapping block inside the generator. Only convert-mode transforms are supported here (no stream-level add or select transforms).

generators:
  department_groups:
    generator_type: local_group
    source_field: department
    name_template: "{{value}}"
    unique_id_template: "dept_{{value}}"
    add_membership: true
    field_mapping:
      department_label:
        source_field: department
        destination_field: description
        transforms:
          - name: jinja_render_convert
            options:
              template: "Department: {{row.department}}"

Reference: Supported Generator Types

local_group

Creates LocalGroup objects. When add_membership: true, calls add_group() on the primary source object (e.g. a LocalUser).

generators:
  dept_groups:
    generator_type: local_group
    source_field: department
    unique_id_template: "dept_{{value}}"
    add_membership: true

local_role

Creates LocalRole objects. When add_membership: true, calls add_role() on the primary source object with apply_to_application=True.

generators:
  title_roles:
    generator_type: local_role
    source_field: job_title
    unique_id_template: "role_{{value}}"
    add_membership: true

Tips and Edge Cases

Deduplication

Generated objects are deduplicated automatically. The runner looks up each rendered unique_id in the provider before creating — if it already exists it reuses it. No configuration is needed.

Normalising values before generation

If your source data has inconsistent casing (e.g. "engineering" vs "Engineering"), apply a lowercase transform to source_field in field_mapping before the generator reads it:

field_mapping:
  department:
    destination_field: department
    transforms:
      - name: lowercase

generators:
  dept_groups:
    generator_type: local_group
    source_field: department          # reads the lowercased value
    unique_id_template: "dept_{{value}}"

Skipping empty values

skip_empty: true (the default) skips any record where source_field is blank or null, preventing objects with empty names from being created. Set skip_empty: false only if you intentionally want to create an object for an empty/zero value.

source_field naming

After field transforms have run, source_field in a generator should match the destination field name from your field_mapping, not the original column name (unless they are the same). In the example above, source_field: department matches the destination_field: department entry in field_mapping.

If the generator’s source_field is not found in field_mapping, a warning is logged but the runner will still attempt to read the field directly from the raw data stream (useful for passthrough columns with no explicit mapping).

Generators with add_membership: false

Set add_membership: false to create groups/roles without assigning membership. This is useful when you want to pre-populate groups for later assignment via a hook or another source.

generators:
  all_departments:
    generator_type: local_group
    source_field: department
    add_membership: false   # create groups but don't assign users