@my-curiosity/react-hand-draw NPM

React Hand Draw

A ReactJS component for drawing using hand movements in the air. Utilizes mediapipe/tfjs handpose detection ML models and react-webcam; requires a visible hand palm on camera.

Installation

npm install @my-curiosity/react-hand-draw

Controls/ How to Use

Basic idea

A canvas pointer corresponds to the index finger and an action is chosen depending on other fingers' distance to it:

middle finger and thumb far away -> draw
middle finger near -> point/ wait
thumb near -> erase

Details

Distances from IFT to IFP, MFT and TT points are calculated, the smallest of them determines action:

IFT - IFP -> draw
IFP - MFT -> point
IFP - TT -> erase

(here IFT = index_finger_tip, IFP = index_finger_pip, MFT = middle_finger_tip, TT = thumb_tip as shown on this diagram)

All relevant points and distances are also shown on top of camera image for clarity. The smallest distance is displayed in green color.

Color Picker

Moving pointer to a color in the color picker at screen edge sets this color as currently active. Color picker can be customized or disabled using props.

Buttons

The component also provides buttons for turning the camera on/off and clearing the entire canvas. Users have to enable the camera manually.

Props

All props are optional, but can be useful for customization.

prop	type	default	notes
webcamFps	number	60	webcam frames/second
webcamMirrored	boolean	true	mirror webcam image
webcamVisible	boolean	true	show webcam in component
controlLightsVisible	boolean	true	show if webcam and detector model are ready using status lights
detectorRuntime	"mediapipe" \| "tfjs"	"mediapipe"	choose a detector model type
canvasColorPickerPosition	"left" \| "right" \| "up" \| "down" \| "off"	"up"	choose color picker position ("off" to disable)
canvasColorPickerSize	number	18	color picker size (default = 1/18 of the canvas)
canvasColors	(string \| CanvasGradient \| CanvasPattern)[]	"red", "pink", "white", "yellow", "green", "blue", "black"	colors available for color picker
canvasDefaultColor	string \| CanvasGradient \| CanvasPattern	"green"	default color