@luxonis/rendering-lib v1.0.3

React WebGl Rendering Library

WebGL

Writing shaders

• esbuild plugin converts .vert and .frag shader files to .js files with string exports
• Each shader pair should be exported from shaders/index.ts for later usage
• Use raczzalan.webgl-glsl-editor VSCode extension to have a syntax highlighting and hints in shader files

Metadata text rendering

• Text rendering is implemented using mSDF method
• Ubuntu Mono (Regular 400) font is used
• Use msdf-bmfont-xml or similar tool to generate font texture with the following parameters:
• • Charset: build-tools/font/charset.txt
• • Format: json
• • Distance: 4
• • Texture size: 512,512 (increase to fit charset into a single image as needed)
• • Font size: 55
• Run json output through config optimizer (required) using build-tools/font/optimize-config.mjs (reads stdin)
• Image texture and config are currently stored as a hardcoded base64 string and JS object to avoid CORS and keep library independent from server (textures/text.ts)

Components

FrameEventCanvas

FrameEventCanvas provides canvas element wrapped with WebGL renderer to render a event frame

// Example usage
<FrameEventCanvas
    imageUrl="/api/frame-event.jpg"
    metadataUrl="/api/frame-event.rh_metadada"
    onError={logger.error}
/>

VideoEventCanvas

VideoEventCanvas provides canvas element wrapped with WebGL renderer to render a video event frame-by-frame
To control player use useVideoEventRendererRef hook

// Example usage
const rendererRef = useVideoEventRendererRef()

const playVideoEvent = React.useCallback(() => {
    logger.info('Playing video')
    rendererRef.current.play();
}, [])

<VideoEventCanvas
    rendererRef={rendererRef}
    videoUrl="/api/video-event.mp4"
    metadataUrl="/api/video-event.rh_metadada"
    onError={logger.error}
    onInvalidMetadata={logger.warn}
/>

<input type="button" value="Play footage" onClick={playVideoEvent} />

VideoStreamCanvas

VideoStreamCanvas provides canvas element wrapped with WebGL renderer to render live stream frame-by-frame

// Example usage
<VideoStreamCanvas
    stream={{ appInstanceId, streamKey }}
    onError={logger.error}
    onStateChange={state => logger.info(`Player state changed to ${state}`)}
/>

TODO

• Remove @luxonis/common-fe dependency