policy_gradient

a2c_learner

Advantage Actor-Critic (A2C) Implementation: Pytorch

class xuance.torch.learners.policy_gradient.a2c_learner.A2C_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

estimate_total_iterations()

Estimated total number of training iterations

update(**samples)

ddpg_learner

Deep Deterministic Policy Gradient (DDPG) Paper link: https://arxiv.org/pdf/1509.02971.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.ddpg_learner.DDPG_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

update(**samples)

mpdqn_learner

Multi-pass parameterised deep Q network (MP-DQN) Paper link: https://arxiv.org/pdf/1905.04388.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.mpdqn_learner.MPDQN_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

update(**samples)

npg_learner

class xuance.torch.learners.policy_gradient.npg_learner.NPG_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

compute_fisher_information(params, obs, act)

update(**samples)

pdqn_learner

Parameterised deep Q network (P-DQN) Paper link: https://arxiv.org/pdf/1810.06394.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.pdqn_learner.PDQN_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

update(**samples)

pg_learner

Policy Gradient (PG) Paper link: https://proceedings.neurips.cc/paper/2001/file/4b86abe48d358ecf194c56c69108433e-Paper.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.pg_learner.PG_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

estimate_total_iterations()

Estimated total number of training iterations

update(**samples)

ppg_learner

Phasic Policy Gradient (PPG) Paper link: http://proceedings.mlr.press/v139/cobbe21a/cobbe21a.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.ppg_learner.PPG_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

estimate_total_iterations()

Estimated total number of training iterations

update(*args)

update_auxiliary(**samples)

update_critic(**samples)

update_policy(**samples)

ppo_learner

Proximal Policy Optimization (PPO) with clip trick Paper link: https://arxiv.org/pdf/1707.06347.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.ppo_learner.PPO_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

estimate_total_iterations()

Estimated total number of training iterations

update(**samples)

ppokl_learner

Proximal Policy Optimization with KL divergence (PPO-KL) Paper link: https://arxiv.org/pdf/1707.06347.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.ppokl_learner.PPOKL_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

estimate_total_iterations()

Estimated total number of training iterations

update(**samples)

sac_learner

Soft Actor-Critic with continuous action spaces (SAC) Paper link: http://proceedings.mlr.press/v80/haarnoja18b/haarnoja18b.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.sac_learner.SAC_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

update(**samples)

sacdis_learner

Soft Actor-Critic with discrete action spaces (SAC-Discrete) Paper link: https://arxiv.org/pdf/1910.07207.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.sacdis_learner.SACDIS_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

update(**samples)

spdqn_learner

Split parameterised deep Q network (SP-DQN) Paper link: https://arxiv.org/pdf/1810.06394.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.spdqn_learner.SPDQN_Learner(config: Namespace, policy: torch.nn.Module)

Bases: Learner

update(**samples)

td3_learner

Twin Delayed Deep Deterministic Policy Gradient (TD3) Paper link: http://proceedings.mlr.press/v80/fujimoto18a/fujimoto18a.pdf Implementation: Pytorch

class xuance.torch.learners.policy_gradient.td3_learner.TD3_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

update(**samples)