policy_gradient

a2c_learner

Advantage Actor-Critic (A2C) Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.a2c_learner.A2C_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

forward_fn(obs_batch, act_batch, adv_batch, ret_batch)

update(**samples)

ddpg_learner

Deep Deterministic Policy Gradient (DDPG) Paper link: https://arxiv.org/pdf/1509.02971.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.ddpg_learner.DDPG_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

forward_fn_actor(obs_batch)

forward_fn_critic(obs_batch, act_batch, next_batch, rew_batch, ter_batch)

update(**samples)

mpdqn_learner

Multi-pass parameterised deep Q network (MP-DQN) Paper link: https://arxiv.org/pdf/1905.04388.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.mpdqn_learner.MPDQN_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

class ConActorNetWithLossCell(*args: Any, **kwargs: Any)

Bases: Cell

construct(x, input_q2)

class QNetWithLossCell(*args: Any, **kwargs: Any)

Bases: Cell

construct(x, dis_a, con_a, label, input_q)

update(obs_batch, act_batch, rew_batch, next_batch, terminal_batch)

npg_learner

class xuance.mindspore.learners.policy_gradient.npg_learner.NPG_Learner(config: Namespace, policy: torch.nn.Module, callback)

Bases: Learner

compute_fisher_information(params, obs, act)

update(**samples)

pdqn_learner

Parameterised deep Q network (P-DQN) Paper link: https://arxiv.org/pdf/1810.06394.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.pdqn_learner.PDQN_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

class ConActorNetWithLossCell(*args: Any, **kwargs: Any)

Bases: Cell

construct(x)

class QNetWithLossCell(*args: Any, **kwargs: Any)

Bases: Cell

construct(x, dis_a, con_a, label)

update(obs_batch, act_batch, rew_batch, next_batch, terminal_batch)

pg_learner

Policy Gradient (PG) Paper link: https://proceedings.neurips.cc/paper/2001/file/4b86abe48d358ecf194c56c69108433e-Paper.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.pg_learner.PG_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

forward_fn(obs_batch, act_batch, ret_batch)

update(**samples)

ppg_learner

Phasic Policy Gradient (PPG) Paper link: http://proceedings.mlr.press/v139/cobbe21a/cobbe21a.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.ppg_learner.PPG_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

forward_fn_auxiliary(*args)

forward_fn_critic(obs_batch, ret_batch)

forward_fn_policy(obs_batch, act_batch, adv_batch, old_log_prob_batch)

update(*args)

update_auxiliary(**samples)

update_critic(**samples)

update_policy(**samples)

ppo_learner

Proximal Policy Optimization (PPO) with clip trick Paper link: https://arxiv.org/pdf/1707.06347.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.ppo_learner.PPO_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

estimate_total_iterations()

Estimated total number of training iterations

forward_fn(obs_batch, act_batch, ret_batch, adv_batch, old_log_prob_batch)

update(**samples)

ppokl_learner

Proximal Policy Optimization with KL divergence (PPO-KL) Paper link: https://arxiv.org/pdf/1707.06347.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.ppokl_learner.PPOKL_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

forward_fn(*args)

update(**samples)

sac_learner

Soft Actor-Critic with continuous action spaces (SAC) Paper link: http://proceedings.mlr.press/v80/haarnoja18b/haarnoja18b.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.sac_learner.SAC_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

forward_fn_actor(x)

forward_fn_alpha(log_pi)

forward_fn_critic(obs_batch, act_batch, rew_batch, next_batch, ter_batch)

update(**samples)

sacdis_learner

Soft Actor-Critic with discrete action spaces (SAC-Discrete) Paper link: https://arxiv.org/pdf/1910.07207.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.sacdis_learner.SACDIS_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

forward_fn_actor(obs_batch)

forward_fn_alpha(log_pi)

forward_fn_critic(obs_batch, act_batch, rew_batch, next_batch, ter_batch)

update(**samples)

spdqn_learner

Split parameterised deep Q network (SP-DQN) Paper link: https://arxiv.org/pdf/1810.06394.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.spdqn_learner.SPDQN_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

class ConActorNetWithLossCell(*args: Any, **kwargs: Any)

Bases: Cell

construct(x, input_q2)

class QNetWithLossCell(*args: Any, **kwargs: Any)

Bases: Cell

construct(x, dis_a, con_a, label, input_q)

update(obs_batch, act_batch, rew_batch, next_batch, terminal_batch)

td3_learner

Twin Delayed Deep Deterministic Policy Gradient (TD3) Paper link: http://proceedings.mlr.press/v80/fujimoto18a/fujimoto18a.pdf Implementation: MindSpore

class xuance.mindspore.learners.policy_gradient.td3_learner.TD3_Learner(config: Namespace, policy: mindspore.nn.Cell, callback)

Bases: Learner

forward_fn_actor(obs_batch)

forward_fn_critic(obs_batch, act_batch, next_batch, rew_batch, ter_batch)

update(**samples)