Service configuration

A collection of all configuration flags and their usages for each DataForge service.

All DataForge services (df-server, df-collector, df-preprocessor, df-renderer, df-deliverer, df-ai-trainer, df-ai-tester, df-ai-runner) can be configured using command-line parameters, a main and override configuration file and environment variables.

When deploying services in Docker, environment variables are the preferred method of configuration.

Order of precedence

The configuration options are applied in this order:

  • Main configuration file (such as df-server.yaml)
  • Override (user) configuration file (such as df-server-user.yaml)
  • Environment variables (such as DF_SERVER_ENCRYPTIONKEY)
  • Command-line options (such as --encryption-key)

Config file names

The main configuration file of each service is simply the service name as a yaml file. For example, the main configuration of the DataForge server is df-server.yaml. The name of the override configuration file is the name of the main configuration file with the suffix -user appended to it. For example, the user configuration file of the DataForge server is df-server-user.yaml.

Configuration key naming

Configuration keys exist across all formats but are transformed. For example, the command line option --encryption-key=example becomes encryptionkey: "example" or alternatively EncryptionKey: "example" when used in a configuration file, since capitalization is ignored. When used as an environment variable, a service prefix is appended. For example, the command line option --encryption-key=example becomes DF_SERVER_ENCRYPTIONKEY=example when used as an environment variable.

List of configuration keys

All DataForge binaries can be run with the --list-environment-keys, --list-config-keys or --list-cli-keys flags to list the appropriate set of configuration keys. You can use the --help option to see the usage and default value for each option. Additionally you can use the --create-empty-config option to print an empty sample configuration to the standard output.

For reference configurations, please check the included docker compose file.

The following is an overview of all configuration flags and their usages. For the configuration options of services not developed by us, please consult the relevant documentation.


Server

Command-line parameters for the service: “DataForge server”.

Collector

Command-line parameters for the service: “DataForge collector”.

Preprocessor

Command-line parameters for the service: “DataForge preprocessor”.

Renderer

Command-line parameters for the service: “DataForge renderer”.

Deliverer

Command-line parameters for the service: “DataForge deliverer”.

AI-trainer

Command-line parameters for the service: “DataForge AI-trainer”.

AI-tester

Command-line parameters for the service: “DataForge AI-tester”.

AI-runner

Command-line parameters for the service: “DataForge AI-runner”.

Last modified January 13, 2025: (dd7487cb4)