@pypestream/pif-compiler NPM

@pypestream/pif-compiler

A TypeScript implementation of the Pypestream Intelligence Framework (PIF) compiler with browser, server, and React components. This package provides tools for parsing, validating, and compiling PIF files into executable conversation flows.

Overview

The PIF Compiler is a tool for compiling Pypestream Intelligence Framework (PIF) files into executable conversation flows. This package provides:

Core Functionality: Parse and compile PIF files with robust error handling
Browser Support: Client-side compilation in browser environments
Server Implementation: Host the compiler as a REST API
Client Library: Interact with the compiler API or perform client-side compilation
React Integration: React components and hooks for easy UI integration

Installation

Since this is a private package, you'll need to be authenticated with the Pypestream npm organization to install it:

# Make sure you're logged in with your Pypestream npm account
npm login --scope=@pypestream

# Install the package
npm install @pypestream/pif-compiler

Or with yarn:

# Make sure you're logged in with your Pypestream npm account
npm login --scope=@pypestream

# Install the package
yarn add @pypestream/pif-compiler

Quick Start

Core Functionality

The core functionality allows you to parse and compile PIF files directly:

import { parsePIFFile, compilePIFData } from '@pypestream/pif-compiler';

// Parse PIF data from a CSV string
const pifString = 'Node Number,Node Type,Node Name,Intent,Entity Type,Entity,NLU Disabled?,Next Nodes,Message,IsStart,Command\n1,D,Start Node,greeting,,,false,2|3,"Hello, welcome!",true,';
const [success, pifData] = parsePIFFile(pifString);

if (success) {
  // Compile the PIF data
  const result = compilePIFData(pifData, { userId: 'test-user' });

  if (result.success) {
    console.log('Compilation successful!');
    console.log('JSON:', result.json);
  } else {
    console.error('Compilation failed:', result.errors);
  }
}

Browser Integration

Client-side Compilation

Compile PIF data directly in the browser without a server:

import { CompilerClient } from '@pypestream/pif-compiler/client';

// Create a client instance (no server URL needed for client-side compilation)
const client = new CompilerClient();

// Compile PIF data locally
const pifString = 'Node Number,Node Type,Node Name,Intent,Entity Type,Entity,NLU Disabled?,Next Nodes,Message,IsStart,Command\n1,D,Start Node,greeting,,,false,2|3,"Hello, welcome!",true,';

// Compile directly in the browser
const result = client.compileLocally(pifString, null, {
  userId: 'browser-user',
  username: 'Browser User'
});

if (result.success) {
  console.log('Client-side compilation successful!');
  console.log('JSON:', result.json);
} else {
  console.error('Client-side compilation failed:', result.errors);
}

File Upload in Browser

import { CompilerClient } from '@pypestream/pif-compiler/client';

// Create a client instance
const client = new CompilerClient();

// Handle file input
document.getElementById('pifFileInput').addEventListener('change', async (event) => {
  const file = event.target.files[0];
  if (!file) return;

  try {
    // Compile the file locally
    const result = await client.compileFileLocally(file, null, {
      userId: 'browser-user'
    });

    if (result.success) {
      console.log('Compilation successful!', result.json);
    } else {
      console.error('Compilation failed:', result.errors);
    }
  } catch (error) {
    console.error('Error:', error);
  }
});

Complete Browser Example

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>PIF Compiler Browser Example</title>
  <script type="module">
    // Import the browser-compatible modules
    import { parsePIFFile, compilePIFData } from '@pypestream/pif-compiler';

    // Function to handle file upload and compilation
    async function handleFileUpload(event) {
      const file = event.target.files[0];
      if (!file) return;

      try {
        // Read the file content
        const content = await file.text();

        // Parse the PIF file
        const [success, pifData] = parsePIFFile(content);

        if (!success) {
          displayResult(`Failed to parse PIF file: ${pifData}`);
          return;
        }

        // Compile the PIF data
        const result = compilePIFData(pifData, { userId: 'browser-user' });

        // Display the result
        if (result.success) {
          displayResult('Compilation successful!', result.json);
        } else {
          displayResult('Compilation failed!', result.errors);
        }
      } catch (error) {
        displayResult(`Error: ${error.message}`);
      }
    }

    // Function to display the result
    function displayResult(message, data = null) {
      const resultElement = document.getElementById('result');
      resultElement.innerHTML = `<p>${message}</p>`;

      if (data) {
        resultElement.innerHTML += `<pre>${JSON.stringify(data, null, 2)}</pre>`;
      }
    }

    // Set up event listener when the DOM is loaded
    document.addEventListener('DOMContentLoaded', () => {
      const fileInput = document.getElementById('pifFile');
      fileInput.addEventListener('change', handleFileUpload);
    });
  </script>
</head>
<body>
  <h1>PIF Compiler Browser Example</h1>

  <div>
    <label for="pifFile">Upload PIF File:</label>
    <input type="file" id="pifFile" accept=".csv,.pif">
  </div>

  <div id="result">
    <p>Upload a PIF file to see the compilation result.</p>
  </div>
</body>
</html>

Server Implementation

This package includes a complete server implementation for compiling PIF data. You can create a standalone server or integrate it with your existing Express application.

Creating a Standalone Server

The simplest way to create a PIF compiler server is to use the startServer function:

import { startServer } from '@pypestream/pif-compiler/server';

// Start the server on port 3001
startServer(3001);

This will create a server with all the necessary routes and middleware for compiling PIF files.

Integrating with an Existing Express App

If you already have an Express application, you can integrate the PIF compiler routes:

import express from 'express';
import { setupRoutes } from '@pypestream/pif-compiler/server';

const app = express();

// Set up middleware (if not already set up)
app.use(express.json());

// Set up PIF compiler routes
setupRoutes(app);

// Start your server
app.listen(3001, () => {
  console.log('Server running on port 3001');
});

Creating a Custom Server

For more control, you can create a custom server with specific configurations:

import express from 'express';
import cors from 'cors';
import { setupRoutes, setupSwagger } from '@pypestream/pif-compiler/server';
import { setupErrorHandler, setupSecurity } from '@pypestream/pif-compiler/server';

// Create Express app
const app = express();

// Configure CORS
app.use(cors({
  origin: ['http://localhost:3000', 'https://your-production-domain.com'],
  methods: ['GET', 'POST'],
  allowedHeaders: ['Content-Type', 'Authorization']
}));

// Set up middleware
app.use(express.json({ limit: '10mb' }));
app.use(express.urlencoded({ extended: true, limit: '10mb' }));

// Set up security middleware
setupSecurity(app);

// Set up PIF compiler routes
setupRoutes(app);

// Set up Swagger documentation
setupSwagger(app);

// Set up error handler (must be last)
setupErrorHandler(app);

// Start server
const PORT = process.env.PORT || 3001;
app.listen(PORT, () => {
  console.log(`PIF Compiler server running on port ${PORT}`);
  console.log(`Swagger documentation available at http://localhost:${PORT}/api-docs`);
});

Server Configuration Options

The server implementation provides several configuration options:

import { startServer } from '@pypestream/pif-compiler/server';

// Start server with custom configuration
startServer(3001, {
  // Enable CORS (default: true)
  cors: true,

  // Configure CORS options
  corsOptions: {
    origin: ['http://localhost:3000'],
    methods: ['GET', 'POST']
  },

  // Enable Swagger documentation (default: true)
  swagger: true,

  // Configure request size limits
  requestSizeLimit: '10mb',

  // Enable security features (default: true)
  security: true,

  // Custom logger function
  logger: (message, level) => {
    console.log(`[${level.toUpperCase()}] ${message}`);
  }
});

API Endpoints

Health Check

GET /health

Returns the health status of the API.

Response:

{
  "status": "ok"
}

Compile PIF Data from JSON

POST /api/compile

Compiles PIF data and optional translation data into a bot JSON structure.

Request Body:

{
  "pifData": "Node Number,Node Type,Node Name,Message,Command,Next Nodes,Condition\n1,S,Start,Hello!,,,\n2,D,Message,How are you?,,,\n3,D,Decision,What would you like to do?,,,\n4,D,End,Goodbye!,,,\n,,,,,,\n1,,,,,2,\n2,,,,,3,\n3,,,,,4,yes\n3,,,,,1,no",
  "translationData": "optional translation data in CSV format",
  "userInfo": {
    "userId": "demo-user",
    "username": "Demo User"
  },
  "graphFeatures": ["feature1", "feature2"]
}

Response:

{
  "success": true,
  "json": {
    "nodes": {
      "1": {
        "id": "1",
        "type": "decision",
        "name": "Start",
        "message": "Hello!"
      },
      "2": {
        "id": "2",
        "type": "decision",
        "name": "Message",
        "message": "How are you?"
      }
    },
    "links": [
      {
        "source": "1",
        "target": "2"
      }
    ],
    "graph": {
      "start_node": "1",
      "global_variables": [],
      "skill_tags": [],
      "accessibility": true,
      "linkable": true,
      "links_use_id": true
    }
  }
}

Compile PIF Data from Files

POST /api/compile-file

Compiles PIF data and optional translation data from uploaded files into a bot JSON structure.

Request Body:

pifFile: PIF file in CSV format
translationFile: Translation file in CSV format (optional)
userInfo: User information as JSON string (optional)
graphFeatures: Graph features to enable as JSON array string (optional)

Response:

Same as the /api/compile endpoint.

API Documentation

The API is documented using Swagger (OpenAPI). When the server is running, you can access the interactive documentation at:

http://localhost:3001/api-docs

This provides a user-friendly interface to explore and test the API endpoints.

Client-Server Integration

The package provides a client that can communicate with a PIF compiler server:

import { CompilerClient } from '@pypestream/pif-compiler/client';

// Create a client instance with server URL
const client = new CompilerClient('http://localhost:3001');

// Compile PIF data using the server
const pifString = 'Node Number,Node Type,Node Name,Intent,Entity Type,Entity,NLU Disabled?,Next Nodes,Message,IsStart,Command\n1,D,Start Node,greeting,,,false,2|3,"Hello, welcome!",true,';

client.compile(pifString, null, {
  userId: 'test-user',
  username: 'Test User'
})
  .then(result => {
    if (result.success) {
      console.log('Server compilation successful!');
      console.log('JSON:', result.json);
    } else {
      console.error('Server compilation failed:', result.errors);
    }
  })
  .catch(error => {
    console.error('Error:', error);
  });

// Compile a file using the server
const fileInput = document.getElementById('pifFileInput');
if (fileInput.files && fileInput.files[0]) {
  client.compileFile(fileInput.files[0])
    .then(result => {
      if (result.success) {
        console.log('File compilation successful!');
        console.log('JSON:', result.json);
      } else {
        console.error('File compilation failed:', result.errors);
      }
    });
}

Browser Integration

This package includes a browser-compatible version that can be used directly in web pages without a build step.

Using via CDN

You can include the package directly in your HTML using a CDN like unpkg:

<script src="https://unpkg.com/@pypestream/pif-compiler/browser-umd"></script>

This will expose a global PifCompiler object with all the necessary functions:

<script>
  document.addEventListener('DOMContentLoaded', () => {
    // Create a client instance
    const client = new PifCompiler.CompilerClient();

    // Compile PIF data locally (client-side)
    const pifData = 'Node Number,Node Type,Node Name,Message,Command,Next Nodes,Condition\n1,S,Start,Hello!,,,\n2,D,Message,How are you?,,,\n3,D,Decision,What would you like to do?,,,\n4,D,End,Goodbye!,,,\n,,,,,,\n1,,,,,2,\n2,,,,,3,\n3,,,,,4,yes\n3,,,,,1,no';

    const result = client.compileLocally(pifData);

    if (result.success) {
      console.log('Compilation successful!', result.json);
    } else {
      console.error('Compilation failed:', result.errors);
    }
  });
</script>

Using via npm

If you're using a build system, you can import the browser version:

// ESM
import { CompilerClient } from '@pypestream/pif-compiler/browser';

// CommonJS
const { CompilerClient } = require('@pypestream/pif-compiler/browser');

Browser Example

A complete browser example is available in the repository at examples/browser-example.html. This example demonstrates:

Text input compilation (server-side)
File upload compilation (server-side)
Local compilation (client-side, no server required)

Client API Reference

CompilerClient

The CompilerClient class provides methods for compiling PIF data both locally and via a server.

import { CompilerClient } from '@pypestream/pif-compiler/client';

// Create a client for local compilation
const localClient = new CompilerClient();

// Create a client for server compilation
const serverClient = new CompilerClient('http://localhost:3001');

Methods

Local Compilation:

compileLocally(pifData, translationData?, userInfo?, graphFeatures?): Compile PIF data locally
compileFileLocally(pifFile, translationFile?, userInfo?, graphFeatures?): Compile a PIF file locally

Server Compilation:

compile(pifData, translationData?, userInfo?, graphFeatures?): Compile PIF data using the server
compileFile(pifFile, translationFile?, userInfo?, graphFeatures?): Compile a PIF file using the server

Utility Functions

The package provides several utility functions for working with PIF data:

import {
  parsePIFFile,
  parseTranslationFile,
  formatErrors,
  isSuccessResult,
  isErrorResult
} from '@pypestream/pif-compiler';

// Parse PIF data
const [success, pifData] = parsePIFFile(pifContent);

// Format errors for display
const errorMessage = formatErrors(result.errors);

// Type guards for better error handling
if (isSuccessResult(result)) {
  // Handle success
} else if (isErrorResult(result)) {
  // Handle error
}

Error Handling

The PIF compiler provides detailed error information when compilation fails. The CompilationResult object has the following structure:

interface CompilationResult {
  success: boolean;
  errors?: CompilationError[];
  graph?: any; // The compiled graph if successful
}

interface CompilationError {
  source: string;
  errors: {
    rowNum: number | null;
    nodeNum: number | null;
    errMsgs: {
      fieldName: string;
      fieldEntry: string | null;
      errorDescription: string;
    }[];
  }[];
}

Troubleshooting

Authentication Issues

If you're having trouble installing the package due to authentication issues:

Make sure you're logged in to npm with your Pypestream account:
```
npm login --scope=@pypestream
```
Verify that you have access to the @pypestream organization:
```
npm org ls pypestream
```
Check your .npmrc file to ensure it has the correct registry configuration:
```
@pypestream:registry=https://registry.npmjs.org/
```

API Connection Issues

If the compiler client can't connect to the API:

Verify that the API URL is correct and accessible
Check for CORS issues if the API is on a different domain
Ensure that the API server is running and properly configured

React Integration

The package includes React components and hooks for easy integration with React applications.

import React, { useState } from 'react';
import {
  PIFCompilerProvider,
  useCompilePIF,
  PIFErrorDisplay
} from '@pypestream/pif-compiler/react';

// Wrap your app with the provider
function App() {
  return (
    <PIFCompilerProvider apiUrl="http://localhost:3001">
      <PIFCompilerComponent />
    </PIFCompilerProvider>
  );
}

// Use the hook in your component
function PIFCompilerComponent() {
  const { compile, isCompiling, result, error } = useCompilePIF();
  const [pifData, setPifData] = useState('');

  const handleCompile = async () => {
    await compile(pifData);
  };

  return (
    <div>
      <textarea
        value={pifData}
        onChange={(e) => setPifData(e.target.value)}
        placeholder="Enter PIF data..."
      />

      <button onClick={handleCompile} disabled={isCompiling}>
        {isCompiling ? 'Compiling...' : 'Compile'}
      </button>

      {error && <div className="error">{error}</div>}

      {result && !result.success && (
        <PIFErrorDisplay errors={result.errors} />
      )}

      {result && result.success && (
        <div>
          <h3>Compilation Successful!</h3>
          <pre>{JSON.stringify(result.json, null, 2)}</pre>
        </div>
      )}
    </div>
  );
}

For more detailed React examples, see the examples/typescript-examples/react-npm-example.tsx file in the repository.