# promptest

> ![v](https://badgen.net/npm/v/promptest) ![license](https://badgen.net/github/license/moekidev/promptest)

Latest version **0.0.5** (published 2023-06-28) · MIT license · 0 weekly downloads

## Install

```sh
npm install promptest
pnpm add promptest
yarn add promptest
bun add promptest
```

## Health

**Score 25/100 (F)** — status: abandoned.

Positive: has types; no vulnerabilities; high quality score.

Warnings: low downloads; no esm support; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.0.5 |
| Published | 2023-06-28 |
| First published | 2023-06-28 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | bundled |
| Module format | CommonJS |
| Node | >=14.0.0 |
| Dependencies | 1 |
| Unpacked size | 19.2 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 0 |
| Maintainers | moekidev |
| Keywords | openai, prompt, ai, chatgpt |

## Links

- npm: https://www.npmjs.com/package/promptest
- Repository: https://github.com/moekidev/promptest
- Homepage: https://github.com/moekidev/promptest#readme
- Issues: https://github.com/moekidev/promptest/issues
- npm.io page: https://npm.io/package/promptest

## Dependencies (1)

- [remeda](https://npm.io/package/remeda.md) ^1.23.0

## Alternatives

- [@expo/fingerprint](https://npm.io/package/@expo/fingerprint.md) — 6.2M weekly downloads
- [@azure/monitor-opentelemetry-exporter](https://npm.io/package/@azure/monitor-opentelemetry-exporter.md) — 850.0K weekly downloads
- [@azure/monitor-opentelemetry](https://npm.io/package/@azure/monitor-opentelemetry.md) — 624.0K weekly downloads
- [@posthog/ai](https://npm.io/package/@posthog/ai.md) — 423.3K weekly downloads
- [fakefilter](https://npm.io/package/fakefilter.md) — 63.9K weekly downloads

## Recent versions

- 0.0.5 (latest) — 2023-06-28
- 0.0.4 — 2023-06-28
- 0.0.3 — 2023-06-28
- 0.0.2 — 2023-06-28
- 0.0.1 — 2023-06-28

## README

# PrompTest

![v](https://badgen.net/npm/v/promptest)
![license](https://badgen.net/github/license/moekidev/promptest)

The Prompt testing library for LLM that allows comparing patterns of prompts.

## Installation

    $ npm i --save-dev promptest

## Usage

```ts
// prompt.test.js
const { Configuration, OpenAIApi } = require("openai")
const { variable, promptest } = require('promptest')

const configuration = new Configuration({
  apiKey: process.env.OPENAI_API_KEY,
})
const openai = new OpenAIApi(configuration)

variable("name", ["John", "Jane"])
variable("age", ["20", "30"])

promptest(
  "Your name is {{name}}, and you are {{age}} years old.",
  "Hello!",
  async (subject, input) => {
    const chatCompletion = await openai.createChatCompletion({
      model: "gpt-3.5-turbo",
      messages: [
        { role: "system", content: subject },
        { role: "user", content: input },
      ],
    })
    return chatCompletion.data.choices[0].message.content
  }
).then(console.log).catch(console.error)
```

```bash
npx node prompt.test.js
```

### Variable

```ts
variable("name", ["John", "Jane"])
```

`variable` is called with variable name and patterns.

### Subject

The subject prompt template is defined with `{{name}}`.

```ts
promptest(
  "Your name is {{name}}, and you are {{age}} years old.",
  // ...
)
```

### Input

Input is ordinary user prompt for testing.

```ts
promptest(
  // ..
  "Hello!"
  // ..
)
```

### Callback

The callback is test function. The arguments are subject prompt and input prompt.

```ts
promptest(
  // ...
  async (subject: string, input: string) => {
    const chatCompletion = await openai.createChatCompletion({
      model: "gpt-3.5-turbo",
      messages: [
        { role: "system", content: subject },
        { role: "user", content: input },
      ],
    })
    return chatCompletion.data.choices[0].message.content
  }
)
```

### Result

The result is JSON.

```json
{
  "input": "Hello!",
  "results": [
    {
      "subject": "Your name is John, and you are 20 years old",
      "output": "Hello! How can I assist you today?"
    },
    {
      "subject": "Your name is Jane, and you are 20 years old",
      "output": "Hello! How can I assist you today?"
    },
    {
      "subject": "Your name is John, and you are 30 years old",
      "output": "Hello! How can I help you today?"
    },
    {
      "subject": "Your name is Jane, and you are 30 years old",
      "output": "Hello! How can I assist you today?"
    },
  ]
}
```

## Development

After checking out the repo, run `npm i` to install dependencies. Then, run `npm run test` to run the tests.

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/moekidev/promptest. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/moekidev/promptest/blob/main/CODE_OF_CONDUCT.md).

## License

The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

## Code of Conduct

Everyone interacting in the Promptest project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/moekidev/promptest/blob/main/CODE_OF_CONDUCT.md).

---
_Source: https://npm.io/package/promptest · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._