@rl-js/interfaces v0.9.5

Interfaces

ActionTraces

Kind: global interface

• ActionTraces
  • .record(state, action) ⇒ ActionTraces
  • .update(error) ⇒ ActionTraces
  • .decay(amount) ⇒ ActionTraces
  • .reset() ⇒ ActionTraces

actionTraces.record(state, action) ⇒ ActionTraces

Records a trace for the given state-action pair.

Kind: instance method of ActionTraces
Returns: ActionTraces - - This object

actionTraces.update(error) ⇒ ActionTraces

Updates the value function based on the stored traces, and the given error.

Kind: instance method of ActionTraces
Returns: ActionTraces - - This object

actionTraces.decay(amount) ⇒ ActionTraces

Decay the traces by the given amount.

Kind: instance method of ActionTraces
Returns: ActionTraces - - This object

actionTraces.reset() ⇒ ActionTraces

Reset the traces to their starting values. Usually called at the beginning of an episode.

Kind: instance method of ActionTraces
Returns: ActionTraces - - This object

ActionValueFunction ⇐ FunctionApproximator

Kind: global interface
Extends: FunctionApproximator

• ActionValueFunction ⇐ FunctionApproximator
  • .call(state, action) ⇒ number
  • .update(state, action, error)
  • .gradient(state, action) ⇒ Array.<number>
  • .getParameters() ⇒ Array.<number>
  • .setParameters(parameters)
  • .updateParameters(errors)

actionValueFunction.call(state, action) ⇒ number

Estimate the expected value of the returns given a specific state-action pair

Kind: instance method of ActionValueFunction
Overrides: call
Returns: number - - The approximated action value (q)

actionValueFunction.update(state, action, error)

Update the value of the function approximator for a given state-action pair

Kind: instance method of ActionValueFunction
Overrides: update

actionValueFunction.gradient(state, action) ⇒ Array.<number>

Compute the gradient of the function approximator for a given state-action pair, with respect to its parameters.

Kind: instance method of ActionValueFunction
Overrides: gradient
Returns: Array.<number> - The gradient of the function approximator with respect to its parameters at the given point

actionValueFunction.getParameters() ⇒ Array.<number>

Get the differentiable parameters of the function approximator

Kind: instance method of ActionValueFunction
Returns: Array.<number> - The parameters that define the function approximator

actionValueFunction.setParameters(parameters)

Set the differentiable parameters fo the function approximator

Kind: instance method of ActionValueFunction

actionValueFunction.updateParameters(errors)

Update the parameters in some direction given by an array of errors.

Kind: instance method of ActionValueFunction

AgentFactory

Kind: global interface

agentFactory.createAgent() ⇒ Agent

Kind: instance method of AgentFactory

Agent

Kind: global interface

• Agent
  • .newEpisode(environment)
  • .act()

agent.newEpisode(environment)

Prepare the agent of the next episode. The Agent should perform any cleanup and setup stepts that are necessary here. An Environment object is passed in, which the agent should store each time.

Kind: instance method of Agent

agent.act()

Perform an action for the current timestep. Usually, the agent should at least: 1) dispatch an action to the environment, and 2) perform any necessary internal updates (e.g. updating the value function).

Kind: instance method of Agent

EnvironmentFactory

Kind: global interface

environmentFactory.createEnvironment() ⇒ Environment

Kind: instance method of EnvironmentFactory

Environment

Kind: global interface

• Environment
  • .dispatch(action)
  • .getObservation() ⇒ *
  • .getReward() ⇒ number
  • .isTerminated() ⇒ boolean

environment.dispatch(action)

Apply an action selected by an Agent to the environment. This could a string representing the action (e.g. "LEFT"), or an array representing the force to apply on actuators, etc.

Kind: instance method of Environment

environment.getObservation() ⇒ *

Get an environment-specific observation for the current timestep. This might be a string identifying the current state, an array representing the current environment parameters, pixel-data representing the agent's vision, etc.

Kind: instance method of Environment
Returns: * - An observation object specific to the environment.

environment.getReward() ⇒ number

Get the reward for the current timestep. Rewards guide the learning of the agent: Positive rewards should be given when the agent selects good actions, and negative rewards should be given when the agent selects bad actions.

Kind: instance method of Environment
Returns: number - A scalar representing the reward for the current timestep.

environment.isTerminated() ⇒ boolean

Return whether or not the current episode is terminated, or finished. For example, this should return True if the agent has reached some goal, if the maximum number of timesteps has been exceeded, or if the agent has otherwise failed. Otherwise, this should return False.

Kind: instance method of Environment
Returns: boolean - A boolean representing whether or not the episode has terminated.

FunctionApproximator

Kind: global interface

• FunctionApproximator
  • .call(args) ⇒ number
  • .update(args, error)
  • .gradient(args) ⇒ Array.<number>
  • .getParameters() ⇒ Array.<number>
  • .setParameters(parameters)
  • .updateParameters(errors)

functionApproximator.call(args) ⇒ number

Call the function approximators with the given arguments. The FA should return an estimate of the value of the function at the point given by the arguments.

Kind: instance method of FunctionApproximator
Returns: number - - The approximated value of the function at the given point

functionApproximator.update(args, error)

Update the value of the function approximator at the given point.

Kind: instance method of FunctionApproximator

functionApproximator.gradient(args) ⇒ Array.<number>

Compute the gradient of the function approximator at the given point, with respect to its parameters.

Kind: instance method of FunctionApproximator
Returns: Array.<number> - The gradient of the function approximator with respect to its parameters at the given point

functionApproximator.getParameters() ⇒ Array.<number>

Get the differentiable parameters of the function approximator

Kind: instance method of FunctionApproximator
Returns: Array.<number> - The parameters that define the function approximator

functionApproximator.setParameters(parameters)

Set the differentiable parameters fo the function approximator

Kind: instance method of FunctionApproximator

functionApproximator.updateParameters(errors)

Update the parameters in some direction given by an array of errors.

Kind: instance method of FunctionApproximator

PolicyTraces

Kind: global interface

• PolicyTraces
  • .record(state, action) ⇒ PolicyTraces
  • .update(error) ⇒ PolicyTraces
  • .decay(amount) ⇒ PolicyTraces
  • .reset() ⇒ PolicyTraces

policyTraces.record(state, action) ⇒ PolicyTraces

Records a trace for the given state-action pair.

Kind: instance method of PolicyTraces
Returns: PolicyTraces - - This object

policyTraces.update(error) ⇒ PolicyTraces

Updates the value function based on the stored traces, and the given error.

Kind: instance method of PolicyTraces
Returns: PolicyTraces - - This object

policyTraces.decay(amount) ⇒ PolicyTraces

Decay the traces by the given amount.

Kind: instance method of PolicyTraces
Returns: PolicyTraces - - This object

policyTraces.reset() ⇒ PolicyTraces

Reset the traces to their starting values. Usually called at the beginning of an episode.

Kind: instance method of PolicyTraces
Returns: PolicyTraces - - This object

Policy

Kind: global interface

• Policy
  • .chooseAction(state) ⇒ *
  • .chooseBestAction(state) ⇒ *
  • .probability(state, action) ⇒ number
  • .update(state, action, error)
  • .gradient(state, action) ⇒ Array.<number>
  • .trueGradient(state, action) ⇒ Array.<number>
  • .getParameters() ⇒ Array.<number>
  • .setParameters(parameters)
  • .updateParameters(errors)

policy.chooseAction(state) ⇒ *

Choose an action given the current state.

Kind: instance method of Policy
Returns: * - An Action object of type specific to the environment

policy.chooseBestAction(state) ⇒ *

Choose the best known action given the current state.

Kind: instance method of Policy
Returns: * - An Action object of type specific to the environment

policy.probability(state, action) ⇒ number

Compute the probability of selecting a given action in a given state.

Kind: instance method of Policy
Returns: number - the probability between 0, 1

policy.update(state, action, error)

Update the probability of choosing a particular action in a particular state. Generally, a positive error should make chosing the action more likely, and a negative error should make chosing the action less likely.

Kind: instance method of Policy

policy.gradient(state, action) ⇒ Array.<number>

Compute the gradient of natural logarithm of the probability of choosing the given action in the given state with respect to the parameters of the policy. This can often be computed more efficiently than the true gradient.

Kind: instance method of Policy
Returns: Array.<number> - The gradient of the policy

policy.trueGradient(state, action) ⇒ Array.<number>

Compute the true gradient of the probability of choosing the given action in the given state with respect to the parameters of the policy. This is contrast to the log gradient which is used for most things.

Kind: instance method of Policy
Returns: Array.<number> - The gradient of log(π(state, action))

policy.getParameters() ⇒ Array.<number>

Get the differentiable parameters of the policy

Kind: instance method of Policy
Returns: Array.<number> - The parameters that define the policy

policy.setParameters(parameters)

Set the differentiable parameters of the policy

Kind: instance method of Policy

policy.updateParameters(errors)

Update the parameters in some direction given by an array of errors.

Kind: instance method of Policy

StateTraces

Kind: global interface

• StateTraces
  • .record(state) ⇒ StateTraces
  • .update(error) ⇒ StateTraces
  • .decay(amount) ⇒ StateTraces
  • .reset() ⇒ StateTraces

stateTraces.record(state) ⇒ StateTraces

Records a trace for the given state

Kind: instance method of StateTraces
Returns: StateTraces - - This object

stateTraces.update(error) ⇒ StateTraces

Updates the value function based on the stored traces, and the given error.

Kind: instance method of StateTraces
Returns: StateTraces - - This object

stateTraces.decay(amount) ⇒ StateTraces

Decay the traces by the given amount.

Kind: instance method of StateTraces
Returns: StateTraces - - This object

stateTraces.reset() ⇒ StateTraces

Reset the traces to their starting values. Usually called at the beginning of an episode.

Kind: instance method of StateTraces
Returns: StateTraces - - This object

StateValueFunction ⇐ FunctionApproximator

Kind: global interface
Extends: FunctionApproximator

• StateValueFunction ⇐ FunctionApproximator
  • .call(state) ⇒ number
  • .update(state, error)
  • .gradient(state) ⇒ Array.<number>
  • .getParameters() ⇒ Array.<number>
  • .setParameters(parameters)
  • .updateParameters(errors)

stateValueFunction.call(state) ⇒ number

Estimate the expected value of the returns given a specific state.

Kind: instance method of StateValueFunction
Overrides: call
Returns: number - - The approximated state value (v)

stateValueFunction.update(state, error)

Update the value of the function approximator for a given state

Kind: instance method of StateValueFunction
Overrides: update

stateValueFunction.gradient(state) ⇒ Array.<number>

Compute the gradient of the function approximator for a given state, with respect to its parameters.

Kind: instance method of StateValueFunction
Overrides: gradient
Returns: Array.<number> - The gradient of the function approximator with respect to its parameters at the given point

stateValueFunction.getParameters() ⇒ Array.<number>

Get the differentiable parameters of the function approximator

Kind: instance method of StateValueFunction
Returns: Array.<number> - The parameters that define the function approximator

stateValueFunction.setParameters(parameters)

Set the differentiable parameters fo the function approximator

Kind: instance method of StateValueFunction

stateValueFunction.updateParameters(errors)

Update the parameters in some direction given by an array of errors.

Kind: instance method of StateValueFunction