1.13.5 • Published 10 months ago

@wowyow/vision-node v1.13.5

Weekly downloads
-
License
ISC
Repository
-
Last release
10 months ago

WowYow Vision Node.js SDK

Getting Started

Examples

SDK Documentation

Data Schema

RTSP


Getting Started

Install

npm install '@wowyow/vision-node'

Include

const Vision = require('@wowyow/vision-node');

Authorize

Vision.APIKey = '';

WowYow API Key can be obtained from WowYow AI Studio.

Initialize

let vision = new Vision('https://www.youtube.com/watch?v=IgiB3dyxUvc');

Examples

YouTube Video URL

MP4 Video URL

MP4 Upload

WebM Upload

RTSP

Documentation

Initialization

There are several ways of initializing the process. As soon as Vision class is instantiated, the processing will begin. Entire process consists of of several subprocesses depending on file type or link. They are: upload, frame extraction & detection.

URL

let vision = new Vision('https://www.youtube.com/watch?v=IgiB3dyxUvc');

Supported sites: YouTube, Dailymotion, Vimeo, Discovery, FOX, Instagram, Facebook & more.

Video Upload

let vision = new Vision('./people_dancing.mp4');

Only mp4 & webm file types are supported.

Configuration

SDK Offers range of configuration options to optimize performance or help debugging. Configuration is passed as second argument in SDK.

OptionTypeDefaultDescription
modelsArray'DETECT_SCENE', 'DETECT_PEOPLE'List of models to use. Available are: DETECT_SCENE, DETECT_PEOPLE, DETECT_CLOTHING
fpsInteger4Processing FPS. Values can be between 4 and 30. This option is ignored if stream is used. See RTSP
targetString''URL of the machine used for processing
directConnectionBooleanfalseSpecify whether or not client should establish direct connection to target
previewBooleanfalseIf true, response data will contain preview images with skeleton for DETECT_PEOPLE model and index numbers for pathing.
logPerfBooleanfalseIf true, performance metrics will be logged to console.
skipFramesBooleantrueThis option is used only for streams. If true, processing will skip as many frames as it's necessary to keep up with video playback. If false, frames will be queued.
footageObjectInclude to optimize detection for specific camera type.
footage.lensString'standard'Type of camera lens. Options are: standard, fisheye, wide
footage.modString''Video modification applied. Options are: dewarped
footage.movementString''Movement of camera. Options are: static, handheld, pan
ffmpegString''If you are experiencing issues with RTSP streams please include path to your own ffmpeg binary via this parameter. If ffmpeg is available in your system PATH, you can just pass 'ffmpeg' as parameter. For more help on ffmpeg installation visit Official FFMPEG Download Page.
Config Example

let vision = new Vision(e, {
  models: ['DETECT_SCENE', 'DETECT_PEOPLE']
});

Events

vision.on('progress', ({ event, progress }) => {}); // events are: upload, frame-extraction & detection.
vision.on('data', (data) => {}); // See Data Schema section for details
vision.on('start', () => {});
vision.on('pause', () => {});
vision.on('resume', () => {});
vision.on('end', () => {});
vision.on('error', (err) => {});
vision.on('metadata', (metadata) => {}); // See Metadata section for details
vision.on('stream', (stream) => {}); // See Stream section for details

Methods

vision.pause();
vision.resume();
vision.stop();

Properties

vision.started;
vision.ended;
vision.paused;
vision.metadata;
vision.stream;

Static Methods

// check if there is an avilable server
let available = Vision.available({
  target: ''
});
console.log(available); // true, false

// fetch complete algorithms
let algorithms = Vision.algorithms({
  target: ''
});
console.log(algorithms); // list of algorithms

// fetch available nodes
let nodes = Vision.nodes({
  target: ''
});
console.log(nodes); // list of nodes available to be used in algorithms

Data Schema

FieldTypeDescriptionNote
dataObjecttop-level object
data.{model}Object
data.{model}.mediaIdStringWowYow Media Identifier
data.{model}.frameIntegerFrame Number
data.{model}.timestampDecimalProcessing FPS.* Not used in RTSP
data.{model}.widthIntegerFrame Width
data.{model}.heightIntegerFrame Height
data.{model}.modelStringModel name
data.{model}.predictionsArrayList of predictions generated by this model.* Check Model Schema(#Model Schema) to see schema for each model.
data.{model}.sourceObject
data.{model}.source.base64StringBase64 link of frame
data.{model}.source.jpegStringJPEG link of frame
data.{model}.previewObject* Included only if preview config is used.
data.{model}.preview.base64StringBase64 link of frame including model overlay information
data.{model}.preview.jpegStringJPEG link of frame including model overlay information

Metadata

FieldTypeDescriptionNote
durationNumberSeconds of video duration. In case of RTSP Stream it will have Infinity value

Stream

FieldTypeDescriptionNote
urlStringUrl to access streamable video file from the server which can be directly added as source of video html element

Model Schema


DETECT_SCENE

FieldTypeDescriptionNote
prediction.indexIntegerScene identifier* Do not expect index to always be incremented by 1

DETECT_PEOPLE

FieldTypeDescriptionNote
predictionObjecttop-level object
prediction.scoreDecimal
prediction.indexIntegerPerson identifier used for pathing* Do not expect index to always be incremented by 1
prediction.keypointsArrayList of keypoints
prediction.keypoints[].partStringName of the body part. Options are: nose, leftEye, rightEye, leftEar, rightEar, leftShoulder, rightShoulder, leftElbow, rightElbow, leftWrist, rightWrist, leftHip, rightHip, leftKnee, rightKnee, leftAnkle, rightAnkle
prediction.keypoints[].positionObject
prediction.keypoints[].position.xDecimalPosition of keypoint on x axis
prediction.keypoints[].position.yDecimalPosition of keypoint on y axis
prediction.segmentsObjectObject containing each person segment
prediction.segments.bodyObject
prediction.segments.body.bboxObjectBounding Box of person body segment
prediction.segments.body.bbox.x0IntegerCoordinate top offset* Check BBox.
prediction.segments.body.bbox.y0IntegerCoordinate left offset* Check BBox.
prediction.segments.body.bbox.x1IntegerCoordinate top offset* Check BBox.
prediction.segments.body.bbox.y1IntegerCoordinate left offset* Check BBox.
prediction.segments.body.previewObject* Included only if preview config is used.
prediction.segments.body.preview.base64ObjectBase64 link of segment cut-out.
prediction.segments.body.preview.jpegObjectJPEG link of segment cut-out.
prediction.segments.faceObject
prediction.segments.face.bboxObjectBounding Box of person face segment
prediction.segments.face.bbox.x0IntegerCoordinate top offset* Check BBox.
prediction.segments.face.bbox.y0IntegerCoordinate left offset* Check BBox.
prediction.segments.face.bbox.x1IntegerCoordinate top offset* Check BBox.
prediction.segments.face.bbox.y1IntegerCoordinate left offset* Check BBox.
prediction.segments.face.previewObject* Included only if preview config is used.
prediction.segments.face.preview.base64ObjectBase64 link of segment cut-out.
prediction.segments.face.preview.jpegObjectJPEG link of segment cut-out.
prediction.positionStringHuman position detection. Options are: sitting, standing
prediction.clothingObjectObject containing clothing* Only if DETECT_CLOTHING model is used. Schema is same as this model's main schema.

DETECT_CLOTHING

FieldTypeDescriptionNote
predictionObjecttop-level object
prediction.scoreDecimal
prediction.indexIntegerClothing identifier used for pathing* Do not expect index to always be incremented by 1
prediction.typeStringType of the clothing. Options are: Leggings, Jodhpurs, Capris, Shorts, Jeans, Joggers, Skirt, Gauchos, Culottes, Sweatshorts, Trunks, Cutoffs, Sarong, Sweatpants, Chinos, Halter, Hoodie, Henley, Parka, Cardigan, Tank, Bomber, Peacoat, Top, Poncho, Button-Down, Anorak, Sweater, Blouse, Turtleneck, Blazer, Jacket, Jersey, Tee, Flannel, Jeggings
prediction.bboxObjectBounding Box of clothing item
prediction.bbox.x0IntegerCoordinate top offset* Check BBox.
prediction.bbox.y0IntegerCoordinate left offset* Check BBox.
prediction.bbox.x1IntegerCoordinate top offset* Check BBox.
prediction.bbox.y1IntegerCoordinate left offset* Check BBox.
prediction.previewObject* Included only if preview config is used.
prediction.preview.base64ObjectBase64 link of segment cut-out.
prediction.preview.jpegObjectJPEG link of segment cut-out.

RTSP

RTSP processing differs from videos with fixed duration. Serveral things need to be kept in mind when using it:

  1. Response data frame and timestamp use start of processing as reference point, starting both from 0.
  2. By default, frames are dropped if processing is lagging behind video playback for any reason. Whether it's because of low processing fps or network issues.

BBox

BBox

1.13.5

10 months ago

1.13.4

11 months ago

1.7.0

2 years ago

1.6.1

2 years ago

1.6.0

2 years ago

1.2.1

2 years ago

1.2.0

2 years ago

1.1.14

2 years ago

1.1.13

2 years ago

1.1.12

2 years ago

1.1.11

2 years ago

1.1.10

2 years ago

1.1.9

2 years ago

1.1.8

2 years ago

1.1.7

2 years ago

1.1.6

2 years ago

1.1.5

2 years ago

1.1.4

2 years ago

1.1.3

2 years ago

1.1.2

2 years ago

1.1.1

2 years ago

1.1.0

2 years ago

1.0.4

2 years ago

1.0.3

2 years ago

1.0.2

2 years ago

1.0.1

2 years ago

1.0.0

2 years ago