|
| This GIF taken from DeepAR offical site. |
DeepAR is an infrastructure where you can make AR applications in ease. DeepAR is not free, but you can create applications that can be used by up to 10 people for testing purposes for free.
- Check pricing from [Pricing Page](https://developer.deepar.ai/pricing).
- In order to use DeepAR, you need to generate an API key. You can generate the API key from the [Developer Panel](https://developer.deepar.ai/).
- With the [Asset Store](https://www.store.deepar.ai/), you can buy ready to use AR content.
- If you're looking for free filters, here is some: [Free Filter Pack](https://docs.deepar.ai/deep-ar-studio/free-filter-pack)
- With the [DeepAR Studio](https://www.deepar.ai/creator-studio), you can create, edit and fine tune your own AR content.
- To learn DeepAR Studio, visit [DeepAR Help Center](https://help.deepar.ai/en/).
**DeepAR Features:**
- [Face Filters, Effects and Masks](https://www.deepar.ai/face-filters-lenses-masks)
- [AR Beauty & Makeup](https://www.deepar.ai/beauty-and-makeup)
- [Realtime Hair Color Segmentation](https://www.deepar.ai/hair-coloring)
- [Background Removal and Segmentation](https://www.deepar.ai/background-removal)
- [Realtime Emotion Detection](https://www.deepar.ai/emotion-detection)
You can visit [DeepAR's offical site](https://www.deepar.ai/) to learn more.
## Compatibility
| DeepAR SDK | lib version (`react-native-deepar`) | Required React Native Version | Android SDK (Min) | iOS Version (Min) |
| ---------- | ----------------------------------- | ----------------------------- | ----------------- | ----------------- |
| 3.4.2 | `>= 0.1.0` && `<= 0.10.2` | `>= 0.64.2` | 21 | 11.0 |
| 3.4.4 | `>= 0.10.3` && `<= 0.10.5` | `>= 0.64.2` | 23 | 11.0 |
| 5.2.0 | `>= 0.11.0` | `>= 0.64.2` | 23 | 11.0 |
## Getting Started
:warning: This library under development, if you found a bug, please open an issue from [here](https://github.com/ridvanaltun/react-native-deepar/issues/new).
:warning: It only works on physical devices, not will work with simulator.
```sh
yarn add react-native-deepar
```
### Other Required Steps
#### Android
1. Open your project's `AndroidManifest.xml` and add the following lines:
```xml
|
| Follow steps in the picture. |
3. Add `DeepAR.xcframework` to Build Phases:
| Add DeepAR to Build Phases (1) | Add DeepAR to Build Phases (2) |
| ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| 
| 
|
| Follow steps in the picture. | Follow steps in the picture. |
**Note:** Don't forget install Pods for iOS and rebuild your app.
### Requesting Permissions
You need to ask necessary permissions before render the DeepAR component.
#### Getting Permissions
Simply use the get functions to find out if a user has granted or denied permission before:
```jsx
import { Camera } from 'react-native-deepar';
// ..
const cameraPermission = await Camera.getCameraPermissionStatus();
const microphonePermission = await Camera.getMicrophonePermissionStatus();
```
A permission status can have the following values:
- `authorized`: Your app is authorized to use said permission. Continue with using the `-
| Called when the DeepAR is initialized. DeepAR methods should not be called before the initialization is completed. |
| onEffectSwitched | (slot: string) | Called when an effect has been switched. |
| onScreenshotTaken | (path: string) | Called when the screen capture is finished. |
| onVideoRecordingPrepared | -
| Called when the video recording is prepared. Check `videoWarmup` option to learn more. **Note:** Only available in iOS |
| onVideoRecordingStarted | -
| The start of the video recording process is not synchronous, so this method will be called when the video recording is started. |
| onVideoRecordingFinished | (path: string) | Called when the video recording is finished. |
| onCameraSwitched | (facing: CameraPositions) | Called when camera switched. |
| onFaceVisibilityChanged | (visible: boolean) | Called when the user's face becomes visible or invisible. |
| onImageVisibilityChanged | (visible: boolean, gameObject?: string) | Called when a natural image is being tracked and the visibility has changed. |
| onError | (text: string, type: ErrorTypes,) | Called when an error occur, like the model path not found or the effect file failed to load. |
### Methods
These are the various methods.
#### General
| Method | Params | Description |
| -------------------- | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| switchEffect | (params: ISwitchEffect) | The method used to switch any effect in the scene. Effects are places in slots. Every slot is identified by its unique name and can hold one effect at any given moment. Every subsequent call to this method removes the effect that was previously displayed in this slot. |
| switchEffectWithPath | (params: ISwitchEffectWithPath) | Same as `switchEffect` but with path. |
| fireTrigger | (trigger: string) | This method allows the user to fire a custom animation trigger for model animations from code. To fire a custom trigger, the trigger string must match the custom trigger set in the Studio when creating the effect. |
| takeScreenshot | -
| Captures a screenshot of the current screen. When a screenshot is done `onScreenshotTaken` will be called with a resulting screenshot. |
| setTouchMode | (enabled: boolean) | This method enable or disable the detection of touches over DeepAR view, it is necessary if your effect has ability to detect touches. |
#### Camera Control
| Method | Params | Description |
| ---------- | ------------------------- | ------------- |
| setFlashOn | (enabled: boolean) | Toggle flash. |
#### Video Recording
| Method | Params | Description |
| --------------- | ------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| startRecording | (params?: [IStartRecording](./docs/interfaces.md#istartrecording)) | Starts video recording of the camera preview. |
| pauseRecording | -
| Pauses video recording. |
| resumeRecording | -
| Resumes video recording after it has been paused with `pauseRecording`. |
| finishRecording | -
| Stops video recording and starts the process of saving the recorded video to the file system. When the file is saved, the method `onVideoRecordingFinished` will be called. |
| setAudioMute | (enabled: boolean) | Mutes/unmutes the audio while video recording. |
#### Change Parameters
For more details about changeParameter API read this article [here](https://help.deepar.ai/en/articles/3732006-changing-filter-parameters-from-code).
| Method | Params | Description |
| ---------------------- | ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| changeParameterFloat | (params: IChangeParamaterFloat) | This method allows the user to change the value of blendshape parameters during runtime. |
| changeParameterVec4 | (params: IChangeParamaterVec4) | This method is used to change the certain color of a Game Object at runtime. |
| changeParameterVec3 | (params: IChangeParamaterVec3) | This method is used to change the transform of a Game Object at runtime, so here you can change the object position, rotation or scale. |
| changeParameterBool | (params: IChangeParamaterBool) | Let say you want to put a button in your app that enables or disables Game Object at runtime. (let's say you want your filter character to put their glasses on or take them off) This function helps you to enable/disable the value. |
| changeParameterString | (params: IChangeParamaterString) | Change a string parameter on a game object. The most common use for this override is to change blend mode and culling mode properties of a game object. **Note:** Only available in iOS |
| changeParameterTexture | (params: IChangeParamaterTexture) | This method allows the user to load an image and set it as a texture during runtime. This can be useful if you want to leverage our background segmentation feature, and change the background in your filter. |
#### Core
| Method | Params | Description |
| --------------------------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| pause | -
| Pauses the rendering. This method will not release any resources and should be used only for temporary pause (e.g. user goes to the next screen). |
| resume | -
| Resumes the rendering if it was previously paused, otherwise doesn't do anything. |
| setLiveMode | (enabled: boolean) | This is an optimization method and it allows the user to indicate the DeepAR in which mode it should operate. If called with true value, DeepAR will expect a continuous flow of new frames and it will optimize its inner processes for such workload. An example of this is the typical use case of processing the frames from the camera stream. If called with false it will optimize for preserving resources and memory by pausing the rendering after each processed frame. A typical use case for this is when the user needs to process just one image. |
| setFaceDetectionSensitivity | (sensitivity: number) | This method allows the user to change face detection sensitivity. The sensitivity parameter can range from 0 to 3, where 0 is the fastest but might not recognize smaller (further away) faces, and 3 is the slowest but will find smaller faces. By default, this parameter is set to 1. |
| showStats | (enabled: boolean) | Display debugging stats on screen. |
### Camera Module
```tsx
import { Camera } from 'react-native-deepar';
```
| Method | Params | Returns | Description |
| ----------------------------- | --------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| requestCameraPermission | -
| Promise<`CameraPermissionRequestResult`> | Shows a "request permission" alert to the user, and resolves with the new camera permission status. If the user has previously blocked the app from using the camera, the alert will not be shown and `"denied"` will be returned. |
| requestMicrophonePermission | -
| Promise<`CameraPermissionRequestResult`> | Shows a "request permission" alert to the user, and resolves with the new microphone permission status. If the user has previously blocked the app from using the microphone, the alert will not be shown and `"denied"` will be returned. |
| getCameraPermissionStatus | -
| Promise<`CameraPermissionStatus`> | Gets the current Camera Permission Status. Check this before mounting the Camera to ensure the user has permitted the app to use the camera. |
| getMicrophonePermissionStatus | -
| Promise<`CameraPermissionStatus`> | Gets the current Microphone-Recording Permission Status. Check this before mounting the Camera to ensure the user has permitted the app to use the microphone. |
## Background Segmentation
DeepAR has [Background Segmentation](https://www.deepar.ai/background-removal) feature, with this feature you can change your background in real-time.
| Background Segmentation Preview |
| ------------------------------------------------------------------------------------------------- |
| 
|
| This image taken from DeepAR offical site. |
There is a filter called `Background` from [Free Filter Pack](https://docs.deepar.ai/deep-ar-studio/free-filter-pack) and you can use this filter.
**How change background image?**
Switch the `Background` effect and apply new background image like below:
```tsx
import { TextureSourceTypes } from 'react-native-deepar';
import RNFetchBlob from 'rn-fetch-blob';
RNFetchBlob.config({})
.fetch('GET', 'https://random.imagecdn.app/450/800')
.then((res) => {
deepARRef?.current?.changeParameterTexture({
gameObject: 'Background',
component: 'MeshRenderer',
parameter: 's_texColor',
type: TextureSourceTypes.BASE64,
value: res.base64(),
});
});
```
## Face Painting
You can detect touches in your effects if the effect customized for detecting touches. For an example; DeepAR has a filter called Face Painting, with this effect you can paint your face in real-time with touches.
For more details about face painting effect read this article [here](https://docs.deepar.ai/guides-and-tutorials/face-paint-texture-tutorial).
| Face Painting Preview |
| ------------------------------------------------------------------------------ |
| 
|
| This image taken from DeepAR offical site. |
If you want to be able to detect touches on the screen, you need to use the following code:
```tsx
// If you switch to the face painting effect, use below code to ability detect touches over DeepAR view
deepARRef?.current?.setTouchMode(true);
// If you not use face painting effect, use below code to disable detecting touches over DeepAR view
deepARRef?.current?.setTouchMode(false);
```
## Watermark
Would you like to add watermark to your filter? Follow this tutorial: [Placing an image as a part of your filter
](https://docs.deepar.ai/guides-and-tutorials/placing-an-image-as-part-of-your-filter)
## Example App
| Ball Face | Background Segmentation |
| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| 
| 
|
```sh
# clone the project
git clone https://github.com/ridvanaltun/react-native-deepar.git
# go into the project
cd react-native-deepar
# make project ready
npm run bootstrap
# go into the example
cd example
# copy environment file and set your api keys (bundle id is com.example.reactnativedeepar)
cp .env.example .env
# run for android
npm run android
# or
# run for ios
npm run ios
```
## Limitations
- Background Segmantation filter only works iPhone 7 and newer devices for iOS
- Hair Segmantation filter not supported on Android and only works iPhone 7 and newer devices for iOS
Learn more about limitations from [features by platform
](https://docs.deepar.ai/deepar-sdk/features-by-platform) page.
## Contributing
See the [contributing guide](CONTRIBUTING.md) to learn how to contribute to the repository and the development workflow.
**This project exists thanks to all the people who contribute.**