How to split a large config.yaml with !include

Large config.yaml files (many sources, connections, long field_mapping blocks) get hard to navigate in one file. OAA runner supports a Home Assistant-style !include tag family so a config can be split across multiple files that are stitched back together at load time.

Tags

  • !include file.yaml — load file.yaml, use its parsed content as the value.

  • !include_dir_list dir/ — list of file contents, one entry per file in dir/ (sorted).

  • !include_dir_named dir/ — dict keyed by filename (without extension); each value is that file’s content.

  • !include_dir_merge_list dir/ — every file in dir/ must contain a list; all lists are concatenated into one list.

  • !include_dir_merge_named dir/ — every file in dir/ must contain a mapping; all mappings are merged (shallow update) into one dict.

Paths are resolved relative to the file doing the including, so nested includes (a file included via !include can itself use !include) work no matter how deep the split goes.

Example

config.yaml
connections.yaml
sources.d/
  users.yaml
  groups.yaml

config.yaml:

version: v2
veza_url: https://example.vezacloud.com
app:
  name: My App
  application_type: Custom App
connections: !include connections.yaml
sources: !include_dir_merge_named sources.d

sources.d/users.yaml:

users:
  destination: LocalUser
  source_type: csv
  csv_filepath: ../data/users.csv

sources.d/groups.yaml:

groups:
  destination: LocalGroup
  source_type: csv
  csv_filepath: ../data/groups.csv

The loaded sources dict ends up identical to writing both blocks inline in config.yaml.

Relative paths inside included files

Fields like mapping_filepath, csv_filepath, and module_directory are still resolved relative to the top-level config file’s directory (Config.BASE_DIR), not the directory of the file that used !include. Keep that in mind when moving a sources: block into a subdirectory — a ../data/foo.csv reference may need an extra ../ once it lives one level deeper.

${VAR} interpolation

Environment variable interpolation (${VAR}) runs after all includes are resolved, so ${VAR} references inside included files work exactly like they do in the main file.

Sanity-checking with explode_config

To see exactly what a split config resolves to, use the explode_config CLI command. It resolves every !include* tag and writes the fully expanded, plain YAML to a file:

oaa explode_config --config_file=./config.yaml
# writes ./config.exploded.yaml

oaa explode_config --config_file=./config.yaml --output=./debug.yaml

The output file has no !include tags left — it’s the same dict that would be handed to Pydantic validation, useful for diffing against a known-good config or debugging a merge.

Limitations

  • !include* tags only work when loading from a local config_file. They are not supported for config_url (remote config) or config_content (in-memory YAML string), since there is no file on disk to resolve relative paths against.

  • Merging tags (!include_dir_merge_list / !include_dir_merge_named) validate that every file in the directory contains the expected type (list or mapping) and raise a clear error otherwise.

  • A missing include file or directory raises FileNotFoundError naming the file that referenced it.