# tabular-sarsa

> A tabular implementation of the SARSA reinforcement learning algorithm which is related to Q-learning

Latest version **1.0.6** (published 2017-05-13) · MIT license · 0 weekly downloads

## Install

```sh
npm install tabular-sarsa
pnpm add tabular-sarsa
yarn add tabular-sarsa
bun add tabular-sarsa
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 1.0.6 |
| Published | 2017-05-13 |
| First published | 2017-05-04 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 0 |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 1 |
| Author | Rod Mcnew |
| Maintainers | rodmcnew |
| Keywords | A, tabular, implementation, of, the, SARSA, reinforcement, learning, algorithm, which, is, related, to, Q-learning, table, based, Q |

## Links

- npm: https://www.npmjs.com/package/tabular-sarsa
- Repository: https://github.com/rodmcnew/tabular-sarsa-js
- Homepage: https://github.com/rodmcnew/tabular-sarsa#readme
- Issues: https://github.com/rodmcnew/tabular-sarsa/issues
- npm.io page: https://npm.io/package/tabular-sarsa

## Alternatives

- [apollo-link-http-common](https://npm.io/package/apollo-link-http-common.md) — 879.0K weekly downloads
- [react-relay](https://npm.io/package/react-relay.md) — 336.8K weekly downloads
- [relay-test-utils](https://npm.io/package/relay-test-utils.md) — 181.6K weekly downloads
- [@vendure/core](https://npm.io/package/@vendure/core.md) — 14.8K weekly downloads
- [@pnpm/deps.graph-sequencer](https://npm.io/package/@pnpm/deps.graph-sequencer.md) — 13.4K weekly downloads

## Recent versions

- 1.0.6 (latest) — 2017-05-13
- 1.0.5 — 2017-05-06
- 1.0.4 — 2017-05-06
- 1.0.3 — 2017-05-06
- 1.0.2 — 2017-05-06
- 1.0.0 — 2017-05-06
- 0.1.3 — 2017-05-05
- 0.1.1 — 2017-05-05
- 0.1.0 — 2017-05-04
- 0.0.1 — 2017-05-04

## README

# Tabular Expected SARSA Agent
This contains an agent that learns to maximize reward through reinforcement learning. The agent works by building a table that can predict the expected value of every possible action from every possible state. Exploration is accomplished by following an epsilon greedy policy.

Because this uses a table-based Q function, it only works in environments with a discrete set of states and actions. You must be able to convert all states and actions to integers to use this agent.

#### Demo:
Select the "Tabular Sarsa" agent here: http://rodmcnew.github.io/reinforcement-learning-agent-tester-js/ 

#### Installation:
```
npm install tabular-sarsa
```

#### Usage:
```Javascript
var agent = new tabularSarsa.Agent(
    numberOfPossibleStates,
    numberOfPossibleActions
);
var lastReward = null;

function tick() {
    /*
     * Tell the agent about the current environment state and
     * have it choose an action to take.
     */
    var action = agent.decide(
        lastReward,
        environment.getCurrentState()
    );

    /*
     * Take the action inside the environment find out how 
     * rewarding the action was.
     */
    lastReward = environment.takeAction(action);
}
```

#### Saving trained agents for later:
```Javascript
//Saving an agent
var agentA = new tabularSarsa.Agent(100, 4);
var savedAgentData = agentA.saveToJson();

//Loading an agent
var agentB = new tabularSarsa.Agent(100, 4);
agentB.loadFromJson(savedAgentData);
```
#### Extra options:
```Javascript
var agent = new tabularSarsa.Agent(
    100,//Number of possible states
    4,//Number of possible actions
    {
        learningEnabled: true,//set to false to disable all learning for higher execution speeds
        learningRate: 0.1,//alpha - how much new experiences overwrite previous ones
        explorationProbability: 0.05,//epsilon - the probability of taking random actions in the Epsilon Greedy policy
        discountFactor: 0.9,//discountFactor - future rewards are multiplied by this
    }
);

```

#### Optimizations beyond plain SARSA that speed up learning:
- Uses "Expected SARSA" rather than plain SARSA
- Uses the first seen reward for each state-action as the initial Q value
 
More info about the Expected-SARSA algorithm:
http://www.cs.ox.ac.uk/people/shimon.whiteson/pubs/vanseijenadprl09.pdf

---
_Source: https://npm.io/package/tabular-sarsa · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._