@lautmaler/jovo-platforms-audiocodes NPM

Jovo plugin library for AudioCodes VoiceAI Connect

This library allows you to build voice apps for the AudioCodes VoiceAI Connect platform with the Jovo Framework. The Audiocodes API documentation is available here

Usage

For context, a typical conversation flow is documented here. As a matter of implementation, Audiocodes-side requests are handled by AudiocodesRequest.ts which maps them to JovoInput objects. Handlers are responsible for processing the input and generating Output objects. This library provides implementations for several Output types in src/output. Finally, AudiocodesPlatformOutputTemplateConverterStrategy maps the output to an AudiocodesResponse and sent back to the Audiocodes platform as JSON.

Supported Output types

stc/output contains all supported Audiocodes output implementations:

MessageActivityOutput: a message type output, used to send a voice prompt to the user.
SessionConfigOutput: configures bot delay options https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/handle-bot-delay.htm?TocPath=Bot%2520integration%257CBasic%2520behavior%257C_5
EndConversationOutput: https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_Combined/hangup.htm
CallTransferOutput: https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_Integration/call-transfer.htm
AbortPromptOutput: https://techdocs.audiocodes.com/voice-ai-connect/Default.htm#VAIG_Combined/aborting-prompts.htm?TocPath=Bot%2520integration%257CControlling%2520the%2520call%257C_8
PlayUrlActivityOutput: https://techdocs.audiocodes.com/voice-ai-connect/#VAIG_Combined/play-url.htm?Highlight=wav
SessionConfigActivityOutput can be used to explicitly configure call session parameters. Defaults are supplied as well. These outputs can be used stand-alone or composed in a single response through Jovo Components.

Using Output types

A help intent handler would look like this:

  @Intents('HelpIntent')
  helpHandler() {
    return this.$send(MessageActivityOutput, {
      text: '<speak> I can demo you some features.</speak>',
    });
  }

The text attribute of MessageActivityOutput is a SSML string which is sent to the user as a voice prompt.

Below is a more complex example where the call is transferred to a human operator. There are two activities sent to the user: a voice prompt using MessageActivityOutput and a call transfer request using CallTransferOutput. The latter has extra activityParams

@Intents(['HumanOperatorIntent'])
humanOperator() {
  this.$send(MessageActivityOutput, {
    text: 'Ok. I will transfer you to a human.',
  });
  this.$send(CallTransferOutput, {
    activityParams: {
      transferTarget: 'sip:some-sip-target-number',
      handoverReason: 'User requests to talk to a human',
    }
  });
  return this.$redirect(CallTransferComponent);
}

Configurable session and activity parameters

Certain session and activity parameters are configurable by controlling attribute values of Output classes. (see Output Types section above for a list of supported output types). A configuration change can be done for the entire call duration via sessionParams, or for the current activity with activityParams.

activityParams and sessionParams Call settings can be adjusted either for the entire session or for individual activities. CallSettingsParams is the data holder for such common attributes and it can be used to set activityParams and/or sessionParams. Here is the exhaustive list of these attributes

language and voiceName (depending on the speech provider) can be set for each message individually. See API doc.
sttSpeechContexts providing hints for improving the accuracy of speech recognitions. See API doc.
sendEventsToBot if specified, sends notification events to the bot. See API doc. By default no events are sent to the bot
enableSpeechInput Enables the activation of the speech-to-text service after the bot message playback is finished. Defaults to true
bargeIn allow user to interrupt when the bot plays a message to the user. default is true
languageDetectionActivate: Activates language detection. Defaults to true.
alternativeLanguages: Defines a list of up to three alternative languages (in addition to the current language) that will be used to detect the language spoken. By default English and German are configured.

DTMF attributes. If sendDTMF is not set, the defaults below are used. Several attributes have defaults in place so they can be omitted if that default value is desired.

sendDTMF: enables collecting DTMF. default is true.
sendDTMFAsText boolean default is false
bargeInOnDTMF boolean default is true
dtmfCollect if false, each DTMF digit is sent on a separate event to the bot. default is true
dtmfCollectInterDigitTimeoutMS: how long to wait before sending all digits to the bot. Default is 2000
dtmfCollectMaxDigits max number of digits collected. Default is 0 (no limit)
dtmfCollectSubmitDigit Defines a special DTMF "submit" digit that when received from the user, VoiceAI sends all the collected digits to the bot. default is #.To disable set to empty string.

Bot delay attributes

botNoInputTimeoutMS: timeout before a prompt is played to the user in case the bot takes more time to process the input . Default is not set.
botNoInputSpeech: textual prompt to play to the user when no input is received from the bot.
botNoInputRetries: maximum number of recurring timeouts for bot response. Default is not set.
botNoInputGiveUpTimeoutMS: timeout response before the call is disconnected. Default is not set.
botNoInputUrl: Audio file to play when the bot does not respond.
resumeRecognitionTimeoutMS: When Barge-In is disabled, speech input is not expected before the bot's response has finished playback. If no reply from the bot arrives within this configured timeout (in milliseconds), VoiceAI Connect expects speech input from the user and speech-to-text recognition is re-activated.

User delay attributes userNoInputTimeoutMS: Maximum time in ms to wait for user input. Defaults to 2000 userNoInputGiveUpTimeoutMS: Maximum time in ms to wait for user input after which the call is terminated. Defaults to 15000 userNoInputRetries: Maximum number of retries to wait for user input. Defaults to 1 or two attempts.

Play URL attributes Attributes configurable via PlayUrlActivityOutput:

playUrlUrl: URL of where the audio file is located.
playUrlMediaFormat: format of the pre-recorded audio file.
playUrlAltText: text to display in the transcript page of the user interface while the pre-recorded audio is played.
playUrlFailureHandling: behaviour in case of failing to play audio: disconnect, notofy-bot and ignore are supported. Use InputType.PLAY_URL_FAILED to handle the event.
playUrlCaching boolean. If true, the audio file is cached on the VoiceAI Connect server. Defaults to true.

Example setting of activityParams when sending a message activity:

@Intents(['HumanOperatorIntent'])
humanOperator() {
  this.$send(MessageActivityOutput, {
    text: 'Frag wie das Wetter ist.',
    activityParams: {
      sttSpeechContexts: [
        {
          phrases: ['Wetter'],
          boost: 20,
        },
        {
          phrases: ['$ADDRESSNUM']
        }
      ],
      sendEventsToBot: ['noUserInput'],
    }
  });

And an example setting of sessionParams when sending a message activity:

@Handle()
welcome() {
  return this.$send(MessageActivityOutput, {
    text: '<speak>Guten Tag.</speak>',
    sessionParams: {
      language: 'de-DE',
      voiceName: 'Marlene',
    }
  });
}

Some activities allow for setting activityParams specific only to them. These are:

handoverReason for call transfers. API doc:
hangupReason when ending a conversation. API doc: There are ActivityParam structures defined for each such activity. They all inherit from CallSettingsParams which allows (re)setting all other common attributes as well.

handoverReason example:

...
const transferActivityParams: TransferActivityParams = {
  ...
  handoverReason: 'User requests to talk to a human'
};
this.$send(CallTransferOutput, { 
  activityParams: transferActivityParams 
});

hangupReason example:

@Intents('EndConversationIntent')
endConversation() {
  const endConversationActivityParams: EndConversationActivityParams = {
    hangupReason: 'Conversation ended successfully',
  };
  return this.$send(EndConversationOutput, {
    text: '<speak> Thanks for calling. Goodbye </speak>',
    activityParams: endConversationActivityParams,
  });
}

Configuring session params All sessionParams like Bot delay and DTMF should be configured using the SessionConfigOutput.sessionParams interface.. The config can be set once and it applies for the entire session. Example:

  @Handle()
  welcome() {
    this.$send(SessionConfigOutput, {
      sessionParams: {
        botNoInputTimeoutMS: 1000,  // timeout before a prompt is played to the user.
        botNoInputSpeech: 'Please wait for bot input.', // textual prompt to play to the user when no input is received from the bot.
        botNoInputGiveUpTimeoutMS: 10000, // timeout response before the call is disconnected.
        botNoInputRetries: 2, // maximum number of recurring timeouts for bot response.
        botNoInputUrl: 'https://audiofile-to-play-when-bot-does-not-respond.url',
        resumeRecognitionTimeoutMS: 1000,
      },
    });
    return this.$send(MessageActivityOutput, {
      text: 'What do you want to try?',
    });
  }

sendEventsToBot session-wide configuration for sending notification events to the bot. Similar to the eventParams attribute.

Handling DTMF input

When configured, the bot can respond to user's DTMF input. The sessionParams.sendDTMF boolean flag (true by default) is responsible for enabling/disabling DTMF collection. The session is configured by default to collect all digits the user inputs and if an input pauses for more than 2sec, the collected digits are sent to the bot.

To handle user's DTMF input a Jovo component has to handle the InputType.DTMF_EVENT event:

  @Handle({ types: [InputType.DTMF_EVENT] })
  handleDTMF() {
    const userDtmf = this.jovo.$input.dtmfDigits;
    if (userDtmf === '9') {
      this.$send(MessageActivityOutput, {
        text: '<speak> Sorry to see you leave.</speak>',
      });
      return this.endConversation();
    } else if (userDtmf === '11') {
      this.$send(MessageActivityOutput, {
        text: '<speak> You typed 11. Win!</speak>',
      });
    }
    ...
  }

Handling Audiocodes events

Audiocodes will send notifications when configured using the sendEventsToBot session or activity param. These events are mapped by the BotNotificationEvent enum to Jovo InputTypes:

InputType.DIALOUT_INITIATED maps dialoutInitiated event
InputType.NO_USER_INPUT maps noUserInput event
InputType.SPEECH_HYPOTHESIS maps speechHypothesis event
InputType.CONVERSATION_END maps conversationEnd event

notifyParamChange.notifyParamChange configures whether to receive notifications when a call param changed. This attribute takes an array of parameters for which VoiceAI Connect notifies the bot whenever their values change. Doc

Usage example:

@Handle({
  types: [InputType.NO_USER_INPUT],
})
handleBotNotificationEvent() {
  return this.$send(MessageActivityOutput, { text: 'Please state your request.' });
}

Check documentation for a detailed description for each notification.

Configuration

AudiocodesPlatform can be configured with the following attributes:

fallbackLanguage: the language to use when the requested language is not supported by the Audiocodes platform. Defaults to empty string.
fallbackTransferPhoneNumber: the fallback phone number to transfer the call if the intended transfer phone number fails. Needs to be preconfigured in the Audiocodes service. Defaults to empty string.
supportedLanguages: a list of locale+voiceName pairs preconfigured in the Audiocodes account. Defaults to empty list.
sessionParams: default session params to use for the entire session. The defaults are the same as in the example below and can be overridden by setting the desired attributes.

app.configure({
  plugins: [
    // Add Jovo plugins here
    new AudiocodesPlatform({
      fallbackLocale: 'en-US',
      transferPhoneNumber: 'sip:+12345@sip-provider',
      supportedLanguages: [
        { locale: 'en-US', voiceName: 'en-US-GuyNeural' },
        { locale: 'de-DE', voiceName: 'de-DE-KatjaNeural' },
      ],
      sessionParams: {
        botNoInputTimeoutMS: 6000,
        botNoInputGiveUpTimeoutMS: 30000,
        botNoInputRetries: 2,
        sendEventsToBot: [
          VoiceAiNotificationEvent.DIALOUT_INITIATED,
          VoiceAiNotificationEvent.NO_USER_INPUT,
          VoiceAiNotificationEvent.CONVERSATION_END,
          VoiceAiNotificationEvent.SPEECH_HYPOTHESIS,
        ],
        userNoInputGiveUpTimeoutMS: 60000,
        notifyParamChange: ['language'],
      },
      plugins: [...]
...

Language handling

The conversation language can be set explicitly by passing a locale in the language attribute of an activity. The locale format is <language code>_<country code>, eg en-US or de-DE. If the desired language is not supported by the Audiocodes configuration, the AudiocodesPlatform config attribute fallbackLanguage is used. In order to set a desired language for an entire user session, set the locale in the AudiocodesResponse.sessionParams.language attribute.

Experimental features

Audiocodes requests handled by the Jovo service are mapped by AudiocodesEventActivityRequest. Audiocodes sends context-specific attributes with the request which are mapped by AudiocodesEventActivityRequest.parameters. Extra attributes can be send as well, depending on the admin configuration of Audiocodes.

For demonstration purposes, an additional request attribute called EventRequestParameters.sipUserToUseris also implemented. This maps the User-to-User SIP header which is sent by the Audiocodes platform during call initiation.

@jovotech/framework @jovotech/output @jovotech/platform-core uuid

@everything-registry/sub-chunk-537

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago