# @quarkworks-inc/avatar-webkit

> Avatar WebKit

Latest version **0.11.0** (published 2022-07-25) · SEE LICENSE IN LICENSE.md license · 0 weekly downloads

## Install

```sh
npm install @quarkworks-inc/avatar-webkit
pnpm add @quarkworks-inc/avatar-webkit
yarn add @quarkworks-inc/avatar-webkit
bun add @quarkworks-inc/avatar-webkit
```

## Health

**Score 30/100 (F)** — status: abandoned.

Positive: has types; esm support; no vulnerabilities; high quality score.

Warnings: low downloads; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.11.0 |
| Published | 2022-07-25 |
| First published | 2022-04-28 |
| Weekly downloads | 0 |
| License | SEE LICENSE IN LICENSE.md |
| TypeScript types | bundled |
| Module format | ESM + CommonJS |
| Dependencies | 10 |
| Unpacked size | 1.2 MB |
| Known vulnerabilities | 0 (+23 in 1 direct dependencies) |
| Install scripts | no |
| GitHub stars | 87 |
| Maintainers | quark-dev |

## Links

- npm: https://www.npmjs.com/package/@quarkworks-inc/avatar-webkit
- Repository: https://github.com/Hallway-Inc/AvatarWebKit
- Homepage: https://github.com/Hallway-Inc/AvatarWebKit#-joinhallwaycom
- Issues: https://github.com/Hallway-Inc/AvatarWebKit/issues
- npm.io page: https://npm.io/package/@quarkworks-inc/avatar-webkit

## Dependencies (10)

- [jose](https://npm.io/package/jose.md) ^4.8.1
- [path](https://npm.io/package/path.md) ^0.12.7
- [rxjs](https://npm.io/package/rxjs.md) 7.0.0
- [axios](https://npm.io/package/axios.md) 0.25.0
- [mathjs](https://npm.io/package/mathjs.md) 10.1.1
- [async-lock](https://npm.io/package/async-lock.md) ^1.3.1
- [bezier-easing](https://npm.io/package/bezier-easing.md) 2.1.0
- [@tensorflow/tfjs](https://npm.io/package/@tensorflow/tfjs.md) 3.1.0
- [@mediapipe/face_mesh](https://npm.io/package/@mediapipe/face_mesh.md) 0.4.1633559619
- [@quarkworks-inc/hallway-protobuffers](https://npm.io/package/@quarkworks-inc/hallway-protobuffers.md) 0.0.8

## Recent versions

- 0.11.0 (latest) — 2022-07-25
- 0.11.0-worker.10 (worker) — 2022-10-13
- 0.11.0-worker.9 — 2022-10-13
- 0.11.0-worker.8 — 2022-10-13
- 0.11.0-worker.7 — 2022-09-01
- 0.11.0-worker.6 — 2022-08-26
- 0.11.0-worker.5 — 2022-08-23
- 0.11.0-worker.4 — 2022-08-22
- 0.11.0-worker.3 — 2022-08-17
- 0.11.0-worker.2 — 2022-08-17
- 0.11.0-worker.1 — 2022-08-17
- 0.11.0-worker.0 — 2022-08-16
- 0.10.1 — 2022-07-18
- 0.10.0 — 2022-07-14
- 0.9.9 — 2022-07-08
- … 14 more at https://npm.io/package/@quarkworks-inc/avatar-webkit/versions

## README

# Avatar WebKit by Hallway

Avatar WebKit uses machine learning models to provide face motion capture blend shapes (aka. "morph targets" or "action units") for animation of human expressions, all in real time in the browser. This can be used in a variety of ways:

- Real-time animation from webcam
- Video calls, AR, VR, Metaverse apps
- Recording high quality animations for movies and games
- Bringing your NFT / personal avatar to life

## Installation

```bash
# yarn
yarn add @quarkworks-inc/avatar-webkit

# npm
npm install @quarkworks-inc/avatar-webkit
```

## First Steps

1. [Get an API token](https://joinhallway.com/sdk/)

2. Start your predictor

```ts
import { AUPredictor } from '@quarkworks-inc/avatar-webkit'
// ...

let predictor = new AUPredictor({
  apiToken: <YOUR_API_TOKEN>,
  shouldMirrorOutput: true,
})

let stream = await navigator.mediaDevices.getUserMedia({
  audio: false,
  video: {
    width: { ideal: 640 },
    height: { ideal: 360 },
    facingMode: 'user'
  }
})

predictor.onPredict = (results => {
  console.log(results)
})

// or if you like RX
predictor.dataStream.subscribe(results => {
  console.log(results)
})

predictor.start({ stream })
```

## Hot Reloading

To get hot reloading / npm link to work for the demo run these steps

1. `cd ~/avatar-webkit`
2. `yarn link`
3. `cd ./demo`
4. `yarn link @quarkworks-inc/avatar-webkit`
5. Delete node modules and yarn.lock in demo folder and in avatar-webkit folder
6. `cd ../`
7. `yarn && yarn start`
8. open a new tab `cd ./demo`
9. `yarn && yarn dev`

## More Docs

https://docs.google.com/document/d/16c3qSYvMi_5l2zXdrsykb2xH6XneOqxTd2wwnEVawxY/edit#

## Benchmarking

### Macbook Pro

16" 2019 MBP, 2.4 GHz Intel Core i9, 16 GB RAM

|         Browser         | Total FPS (Avg Frame) | face mesh (Avg Frame) | Dense Predictor (Avg Frame) |
| :---------------------: | :-------------------: | :-------------------: | :-------------------------: |
| Chrome - (2560 × 1440)  |    52 FPS (19 ms)     |    96 FPS (9.4ms)     |      104 FPS (9.6 ms)       |
|     Chrome - Retina     |    50 FPS (20 ms)     |    98 FPS (10.2ms)    |      102 FPS (9.8 ms)       |
|  Brave - (2560 × 1440)  |    54 FPS (19 ms)     |    103 FPS (9.7ms)    |      107 FPS (9.3 ms)       |
|     Brave - Retina      |    50 FPS (20 ms)     |    95 FPS (10.5ms)    |      105 FPS (9.5 ms)       |
| Safari - (2560 × 1440)  |    30 FPS (33 ms)     |   57 FPS (17.6 ms)    |      64 FPS (15.7 ms)       |
|     Safari - Retina     |    30 FPS (33 ms)     |   57 FPS (17.6 ms)    |      64 FPS (15.7 ms)       |
| Firefox - (2560 × 1440) |    35 FPS (29 ms)     |   71 FPS (14.1 ms)    |      63 FPS (15.9 ms)       |
|    Firefox - Retina     |    36 FPS (30 ms)     |    74 FPS (13.6ms)    |      61 FPS (16.4 ms)       |
|  Opera - (2560 × 1440)  |    48 FPS (21 ms)     |    91 FPS (11 ms)     |       100 FPS (10 ms)       |
|     Opera - Retina      |    45 FPS (22 ms)     |   85 FPS (11.8 ms)    |      98 FPS (10.2 ms)       |

### iPhone XR

| Browser | Total FPS (Avg Frame) | face mesh (Avg Frame) | Dense Predictor (Avg Frame) |
| :-----: | :-------------------: | :-------------------: | :-------------------------: |
| Chrome  |    13 FPS (77 ms)     |                       |                             |
| Safari  |    13 FPS (77 ms)     |                       |                             |

---
_Source: https://npm.io/package/@quarkworks-inc/avatar-webkit · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._