Model-Based Reinforcement Learning

In this lesson, we assume that the agent has a model of the environment, which includes both the knowledge of reward states $\rho(s,a)$ and the transfer functions $\tau(s,a)$. The knowledge of these functions, or a model thereof, is required for model-based $\text{RL}$. The basic challenge in practical applications is to learn these functions from examples of the agent acting in the environment. Here, we are more concerned with showing how to calculate optimal policies if we know these functions.

The basic Bellman equation

Below, we have again the state-action value function: