Building a Machine Learning Pipeline from Scratch/

...

/

Configuration Files: OmegaConf

Configuration Files: OmegaConf

Explore config files and the OmegaConf library we use to handle them.

We'll cover the following...

What are configuration files?

Configuration files, or config files, are auxiliary files in a project that control the initial settings, features, and parameters of a program. We can think of them as a file-based user interface. Instead of choosing from drop-downs or clicking radio buttons, we enter our settings in a file. For example, a config file for an ML pipeline can have an entry for data path or test size. Config files can be in one of a variety of formats, such as YAML, TOML, or INI. Let’s take a look at a config file in YAML format.

Press + to interact
# path to data
data_path: /data/iris.csv
# use 10% of the data for evaluation
test_split: 0.1
# features to use in training
features:
  - sepal_length
  - sepal_width
  - petal_length
  - petal_width
# target variable
target: species
# pipeline tasks
tasks:
  tasks:
    load_data:
      next:
        - clean_data
    clean_data:
      next:
        - train_model
    train_model:
      next:
        - evaluate_model
    evaluate_model:
      next: []

In this example we define the path to the data in data_path. We then define the size of the test fraction in test_split, indicating that 10% of the data is to be used for evaluation. This config file corresponds to the iris classification problem. We see four features corresponding to the iris dataset under the features entry. The ...