@imgly/plugin-ai-video-generation-web NPM

IMG.LY AI Video Generation for Web

A plugin for integrating AI video generation capabilities into CreativeEditor SDK.

Overview

The @imgly/plugin-ai-video-generation-web package enables users to generate videos using AI directly within CreativeEditor SDK. This shipped provider leverages the fal.ai platform to provide high-quality video generation from text-to-video and image-to-video transformations.

Features include:

Text-to-video generation
Image-to-video transformations
Multiple model options
Automatic history tracking
Canvas menu quick actions
Seamless integration with CreativeEditor SDK

Installation

npm install @imgly/plugin-ai-video-generation-web

Usage

Basic Configuration

To use the plugin, import it and configure it with your preferred providers:

import CreativeEditorSDK from '@cesdk/cesdk-js';
import VideoGeneration from '@imgly/plugin-ai-video-generation-web';
import FalAiVideo from '@imgly/plugin-ai-video-generation-web/fal-ai';

// Initialize CreativeEditor SDK
CreativeEditorSDK.create(domElement, {
    license: 'your-license-key'
    // Other configuration options...
}).then(async (cesdk) => {
    // Add the video generation plugin
    cesdk.addPlugin(
        VideoGeneration({
            // Text-to-video provider
            text2video: FalAiVideo.MinimaxVideo01Live({
                proxyUrl: 'https://your-fal-ai-proxy.example.com'
            }),

            // Image-to-video provider (optional)
            image2video: FalAiVideo.MinimaxVideo01LiveImageToVideo({
                proxyUrl: 'https://your-fal-ai-proxy.example.com'
            }),

            // Optional configuration
            debug: false,
            dryRun: false
        })
    );
});

Providers

The plugin comes with pre-configured providers for fal.ai models:

1. MinimaxVideo01Live (Text-to-Video)

A model that generates videos based on text prompts:

text2video: FalAiVideo.MinimaxVideo01Live({
    proxyUrl: 'https://your-fal-ai-proxy.example.com'
});

Key features:

Generate videos from text descriptions
Fixed output dimensions (1280×720)
5-second video duration

2. MinimaxVideo01LiveImageToVideo (Image-to-Video)

A model that transforms still images into videos:

image2video: FalAiVideo.MinimaxVideo01LiveImageToVideo({
    proxyUrl: 'https://your-fal-ai-proxy.example.com'
});

Key features:

Transform existing images into videos
Available through canvas quick actions
Maintains original image aspect ratio

3. PixverseV35TextToVideo (Text-to-Video)

An alternative text-to-video model:

text2video: FalAiVideo.PixverseV35TextToVideo({
    proxyUrl: 'https://your-fal-ai-proxy.example.com'
});

Configuration Options

The plugin accepts the following configuration options:

Option	Type	Description	Default
`text2video`	Provider	Provider for text-to-video generation	undefined
`image2video`	Provider	Provider for image-to-video transformation	undefined
`debug`	boolean	Enable debug logging	false
`dryRun`	boolean	Simulate generation without API calls	false
`middleware`	Function[]	Array of middleware functions for the generation	undefined

Middleware Configuration

The middleware option allows you to add pre-processing and post-processing capabilities to the generation process:

import VideoGeneration from '@imgly/plugin-ai-video-generation-web';
import FalAiVideo from '@imgly/plugin-ai-video-generation-web/fal-ai';
import { loggingMiddleware, rateLimitMiddleware } from '@imgly/plugin-ai-generation-web';

// Create middleware functions
const logging = loggingMiddleware();
const rateLimit = rateLimitMiddleware({
  maxRequests: 5,
  timeWindowMs: 300000, // 5 minutes
  onRateLimitExceeded: (input, options, info) => {
    console.log(`Video generation rate limit exceeded: ${info.currentCount}/${info.maxRequests}`);
    return false; // Reject request
  }
});

// Create custom middleware
const customMiddleware = async (input, options, next) => {
  console.log('Before generation:', input);
  
  // Add custom fields or modify the input
  const modifiedInput = {
    ...input,
    customField: 'custom value'
  };
  
  // Call the next middleware or generation function
  const result = await next(modifiedInput, options);
  
  console.log('After generation:', result);
  
  // You can also modify the result before returning it
  return result;
};

// Apply middleware to plugin
cesdk.addPlugin(
  VideoGeneration({
    text2video: FalAiVideo.MinimaxVideo01Live({
      proxyUrl: 'https://your-fal-ai-proxy.example.com'
    }),
    middleware: [logging, rateLimit, customMiddleware] // Apply middleware in order
  })
);

Built-in middleware options:

loggingMiddleware: Logs generation requests and responses
rateLimitMiddleware: Limits the number of generation requests in a time window

You can also create custom middleware functions to meet your specific needs.

Using a Proxy

For security reasons, it's recommended to use a proxy server to handle API requests to fal.ai. The proxy URL is required when configuring providers:

text2video: FalAiVideo.MinimaxVideo01Live({
    proxyUrl: 'https://your-fal-ai-proxy.example.com'
});

You'll need to implement a proxy server that forwards requests to fal.ai and handles authentication.

API Reference

Main Plugin

VideoGeneration(options: PluginConfiguration): EditorPlugin

Creates and returns a plugin that can be added to CreativeEditor SDK.

Plugin Configuration

interface PluginConfiguration {
    // Provider for text-to-video generation
    text2video?: AiVideoProvider;

    // Provider for image-to-video generation
    image2video?: AiVideoProvider;

    // Enable debug logging
    debug?: boolean;

    // Skip actual API calls for testing
    dryRun?: boolean;

    // Extend the generation process
    middleware?: GenerationMiddleware;
}

Fal.ai Providers

MinimaxVideo01Live

FalAiVideo.MinimaxVideo01Live(config: {
  proxyUrl: string;
  debug?: boolean;
}): AiVideoProvider

MinimaxVideo01LiveImageToVideo

FalAiVideo.MinimaxVideo01LiveImageToVideo(config: {
  proxyUrl: string;
  debug?: boolean;
}): AiVideoProvider

PixverseV35TextToVideo

FalAiVideo.PixverseV35TextToVideo(config: {
  proxyUrl: string;
  debug?: boolean;
}): AiVideoProvider

UI Integration

The plugin automatically registers the following UI components:

Generation Panel: A sidebar panel for text-to-video generation
Quick Actions: Canvas menu items for image-to-video transformations
History Library: Displays previously generated videos
Dock Component: A button in the dock area to open the video generation panel

Panel IDs

Main panel: ly.img.ai/video-generation
Canvas quick actions: ly.img.ai.video.canvasMenu
Provider-specific panels:
- MinimaxVideo01Live: ly.img.ai/fal-ai/minimax/video-01-live
- MinimaxVideo01LiveImageToVideo: ly.img.ai/fal-ai/minimax/video-01-live/image-to-video
- PixverseV35TextToVideo: ly.img.ai/fal-ai/pixverse/v3.5/text-to-video

Asset History

Generated videos are automatically stored in asset sources with the following IDs:

MinimaxVideo01Live: fal-ai/minimax/video-01-live.history
MinimaxVideo01LiveImageToVideo: fal-ai/minimax/video-01-live/image-to-video.history
PixverseV35TextToVideo: fal-ai/pixverse/v3.5/text-to-video.history

Dock Integration

The plugin automatically registers a dock component with a sparkle icon that opens the video generation panel. To customize the component's position in the dock, use the setDockOrder method:

// Add the AI Video component to the beginning of the dock
cesdk.ui.setDockOrder([
  'ly.img.ai/video-generation.dock',
  ...cesdk.ui.getDockOrder()
]);

// Or add it at a specific position
const currentOrder = cesdk.ui.getDockOrder();
currentOrder.splice(2, 0, 'ly.img.ai/video-generation.dock');
cesdk.ui.setDockOrder(currentOrder);