Simple CSV Integration

This configuration tells OAA to load a file called data/sample_data_2.csv. This configuration is the runner version of the community simple csv integration.

The config.yaml file

download this file

version: v2
app:
  name: Demo - v2
  application_type: DemoV2
module_directory: modules
logic_modules:
- simple_csv
sources:
  users:
    destination: LocalUser
    source_type: csv
    csv_filepath: sample_data_2.csv
    field_mapping:
      USER_ID:
        destination_field: unique_id
      USER_NAME:
          destination_field: name
      EMAIL:
        destination_field: email
        custom_type: STRING
      IS_ACTIVE:
        destination_field: is_active
        transforms:
          - name: validate_boolean
      LAST_LOGIN:
        destination_field: last_login_at
        transforms:
          - name: date_convert
      LAST_LOGIN-to-created-at:
        destination_field: last_login_at
        source_field: LAST_LOGIN
        transforms:
          - name: date_convert
  roles:
    destination: LocalRole
    source_type: source
    source: users
    field_mapping:
      ROLE:
        destination_field: unique_id
      ROLE-to-name:
          destination_field: name
          source_field: ROLE
    record_transforms_non_stream:
      - user_to_role

The modules file

modules/simple_csv.py

download this file

# modules/simple_csv.py

from oaa.hooks.decorators import record_transform
from oaa.hooks.decorators import OAAHookEvent


@record_transform(stream=False)
def user_to_role(record, provider=None, **kwargs):
    user_ref = provider.local_users[record.USER_ID]
    user_ref.add_role(record.ROLE, apply_to_application=True)

sample_data_2.csv

download this file

USER_ID,USER_NAME,EMAIL,IS_ACTIVE,LAST_LOGIN,ROLE
10001,Adam Thompson,athompson@example.com,1,2023-10-20 13:05:34,Admin
10001,Adam Thompson,athompson@example,1,2023-10-20 13:05:34,User
10003,Bob Smith,bsmith@example,0,2023-03-20 03:34:23,User