Deep Reinforcement Learning

Value-function approximation with ANN

Up to this point, we have outlined the basic ideas behind reinforcement learning algorithms. We will now move on to an important topic to scale these ideas to real-world applications. In the previous method, we used tabulated values for the functions. For example, the value function in the previous programs was composed of look-up tables or arrays in programming terms that specified the value function for each discrete state and action.

Correspondingly, this leads to tables for the policy. Such algorithms are now commonly referred to as tabular $\textbf{RL}$ algorithms. The problem with this approach is that these tables can be very big for systems with large state and action spaces. Indeed, the increased computational demand of calculating these quantities in many real-world applications is often prohibitive, in particular in a stochastic setting where we have to sample in the state-action space.