Likelihood Ratio Policy Gradient via Importance Sampling

Connection between Likelihood ratio policy gradient method and Importance sampling method. ...

May 25, 2022 · 5 min · Trung H. Nguyen

Eligible Traces

Beside $n$-step TD methods, there is another mechanism called eligible traces that unify TD and Monte Carlo. Setting $\lambda$ in TD($\lambda$) from $0$ to $1$, we end up with a spectrum ranging from TD methods, when $\lambda=0$ to Monte Carlo methods with $\lambda=1$. ...

March 13, 2022 · 25 min · Trung H. Nguyen

Function Approximation

All of the tabular methods we have been considering so far might scale well within a small state space. However, when dealing with Reinforcement Learning problems in continuous state space, an exact solution is nearly impossible to find. But instead, an approximated answer could be found. ...

February 11, 2022 · 21 min · Trung H. Nguyen

Temporal-Difference Learning

So far in this series, we have gone through the ideas of dynamic programming (DP) and Monte Carlo. What will happen if we combine these ideas together? Temporal-difference (TD) learning is our answer. ...

January 31, 2022 · 21 min · Trung H. Nguyen

Monte Carlo Methods in Reinforcement Learning

Recall that when using Dynamic Programming algorithms to solve RL problems, we made an assumption about the complete knowledge of the environment. With Monte Carlo methods, we only require experience - sample sequences of states, actions, and rewards from simulated or real interaction with an environment. ...

August 21, 2021 · 20 min · Trung H. Nguyen