openai-text-splitter v1.2.1

OpenAI Text Splitter

Openai prompt splitter. Dividing large texts into manageable parts for processing with OpenAI APIs. A library inspired by chatgpt-prompt-splitter to divide large texts into smaller parts for efficient processing with OpenAI APIs.

Installation

  npm install openai-text-splitter

Usage

Arguments

Node: Configure textChunkLength and tokenLimitPerPrompt according to content you are processing.

import {splitTextForOpenAI} from 'openai-text-splitter';

const text = 'Your large text here...';
const textChunkLength = 500;
const tokenLimitPerPrompt = 2048;

const messages = splitTextForOpenAI(text, {
  textChunkLength,
  tokenLimitPerPrompt,
});