Hyperparameters to Train a Bayesian Network

In this lesson, we explore the essential structural criteria to consider when constructing causal models based on Bayesian networks. Our primary focus will be on the model's structure and the concept of hyperparameters, which are expert-dependent parameters that need to be set before running the learning algorithms.

For instance, hyperparameters are adjustable, high-level parameters that govern the behavior of the learning process and the overall architecture of the model. They are typically set before training the model and are not learned during the training process. The choice of hyperparameters can significantly impact the model's performance.

Hyperparameters in Bayesian networks

Hyperparameters rely on the available data as well as expert knowledge. As a result, the challenge extends beyond parameter optimization and encompasses the setting of hyperparameters. Selecting the optimal configuration of hyperparameters in BNs can be a complex task, as these parameters are grounded in the model's semantics. It is crucial that the chosen hyperparameters are pertinent to the data used for training the model and the expert's knowledge. Some common hyperparameters in BNs include:

• Structure-based hyperparameters

• Data based hyperparameters

Structure-based hyperparameters

Structure-based hyperparameters in BNs refer to those parameters that determine the overall structure and organization of the network. The structure of a BN is represented as a DAG with nodes and directed edges. These structure-based hyperparameters include:

1. Network structure: The arrangement of nodes and the directed edges between them, representing the causal relationships among variables.

2. The number of input nodes: Input nodes represent the variables that serve as the starting points of the BN, which are often the observable variables in a dataset. The number of input nodes determines the initial complexity of the network and impacts the relationships between other nodes in the BN.

3. The number of states of each input node: Each input node can have multiple states, representing the different possible values or categories the variable can take. The number of states for each input node affects the conditional probability distributions in the BN and the overall complexity of the network, as more states usually lead to more extensive probability tables.

4. The number of synthetic nodes: Synthetic or intermediate nodes represent variables that lie between the input and target nodes in the network. These nodes can capture the causal relationships between variables, helping to model complex dependencies and interactions in the data. The number of synthetic nodes affects the depth and complexity of the BN and can have a significant impact on the model's performance and interpretability.

5. The number of states of the target node(s): Target nodes are the variables of interest in the BN, often representing the outcome or goal of the model. Like input nodes, target nodes can have multiple states, which impacts the conditional probability distributions and the complexity of the network. The number of states of the target node(s) is crucial in determining the granularity of the predictions and the relationships between the target nodes and other nodes in the BN.

In the following example, we can see a Bayesian network with its structural hyperparameters