Integrate Phone Calls to a Dialogflow Chatbot Via WebSockets

Pre-requisites

  1. A Dialogflow account
  2. A Vonage API Account

Create a Dialogflow Agent

Follow the steps

  1. Open the dialogflow console
  2. Create an agent by giving it a name, setting the default language, choosing the timezone, and clicking on the button to create an agent. The button can either be shown as Create or Create Agent
  3. On the menu on the left click on the gear icon
  4. Under Google Project, click on the project ID name. For instance, VonageDF This will take you to the Google Cloud console

Create a Service Account on the Google Cloud Console

Follow the steps

  1. From the Google Cloud console, click on Go to project settings
  2. On the menu on the left, click on Service accounts
  3. At the top bar, click on + Create Service Account
  4. Give it a Service Account Name you’ll remember, for instance VonageDF
  5. Add a service account description
  6. Click on Create and Continue
  7. Under Grant this service account access to project, filter and select the role Dialogflow API Admin
  1. Click on the three dots under action
  2. Click on Manage keys
  3. Click on Add Key
  4. Click on Create new Key
  5. Select JSON
  6. Click on Create
  7. You can see one .json file was downloaded. We will add this file to the Dialogflow Reference Connection GitHub repository we will clone in the next step

Set Up the Dialogflow Reference Connection

Follow the steps

  1. Clone the dialogflow-reference-connection repository from GitHub and change the directorygit clone https://github.com/nexmo-community/dialogflow-reference-connection && cd dialogflow-reference-connection
  2. Open the code in your favorite IDE or text editor
  3. Add the .json file just downloaded to the root of the project
  4. Create a .env file from the .env.example
  5. Populate the environment variables: GCLOUD_PROJECT_ID=<TheProjectIdFoundOnDialogflow>GOOGLE_CLOUD_CREDENTIALS=<TheJSONFileYouDownloaded.json> From the terminal:
  6. Install the dependencies npm install
  7. Run the file Dialogflow connecting server node df-connecting-server.js There are many deployment types, such as App Engine, Heroku, Cloud Run.
  1. Run ngrok http 5000
  2. Make a note of the URL found in the console (e.g. xxxx.ngrok.io). We will add this information in the DF_CONNECTING_SERVER argument of Set Up the Dialogflow Voice API Sample Application' part of the tutorial
  1. Install git
  2. Install Heroku command line and login to your Heroku account
  3. If you do not yet have a local git repository, create one: git init
  4. Start by creating this application on Heroku from the command line using the Heroku CLI. Note: In the below command, replace thisappname with a unique name on the whole Heroku platform. heroku create thisappname
  5. On your Heroku dashboard where your application page is shown, click on Settings button, add the following Config Vars and set them with their respective values:
  1. Deploy the application git push heroku master or git push heroku main depending on your branch name.
  2. Make a note of the URL found in the console (e.g. thisappname.herokuapp.com). We will add this information in the DF_CONNECTING_SERVER argument of 'Set Up the Dialogflow Voice API Sample Application' part of the tutorial

Set Up Your Vonage Voice API Application Credentials and Phone Number

  1. Create an Application from the Vonage API Dashboard
  2. Give the application a name, for instance VonageDF
  3. Click on the button to generate a public and private key, a private.key file will be downloaded. We will add it to the Dialogflow Voice API Sample Application in the next section of this tutorial
  4. Link a phone number to this application. If you do not already have one, on the menu on the left click on Numbers and then Buy Numbers and follow the steps to purchase it

Set Up the Dialogflow Voice API Sample Application

  • Send audio to the Dialogflow agent from caller’s speech,
  • Stream audio responses from the Dialogflow agent to the caller via the WebSocket,
  • Post back in real-time transcripts and caller’s speech sentiment scores via webhooks callbacks to this Voice API Sample Application.

Follow the steps

  1. From your terminal, clone the Dialogflow Voice API Sample Application from GitHub and change the directorygit clone https://github.com/nexmo-community/dialogflow-sample-voice-application && cd dialogflow-sample-voice-application You can continue to follow the below steps explanation or from the readme file of theDialogflow Voice API Sample Application
  2. Add the private.key generated from the Vonage Dashboard and downloaded to your machine to the project root
  3. From the .env.example create a .env file
  4. Populate the environment variables with the information present on the Vonage API Dashboard
  • The SERVICE_NUMBER is the virtual number you purchased. Remember to add the country code without any 00s or + beforehand
  • The DF_CONNECTING_SERVER is the server from the Dialogflow Reference you already have running (the one you previously took note of)
  1. Install the dependencies npm install
  2. Run the df-application.jsnode df-application.js Below you can find the next steps for ngrok and for Heroku respectfully:
  1. On a separate terminal tab run ngrok http 8000 Get back to the Vonage Dashboard website and under capabilities:
  2. Toggle Voice to enable this capability
  3. Add the ngrok URL running on dialogflow-sample-voice-application followed by /answer on the Answer URL. Make sure HTTP GET is selected.
  4. Add the ngrok URL running on dialogflow-sample-voice-application followed by /event on the Event URL. Make sure HTTP POST is selected.
  5. Click on Save Changes
  1. On a separate terminal tab, If you do not yet have a local git repository, create one: git init
  2. Start by creating this application on Heroku from the command line using the Heroku CLI. Note: In command, replace myappname with a unique name on the whole Heroku platform. heroku create myappname
  3. On your Heroku dashboard where your application page is shown, click on Settings button, add the following Config Vars and set them with their respective values found in your .env file.
  1. On your Heroku dashboard where your application page is shown, click on the Open App button and copy the URL
  2. Now, let’s deploy the application. Get back to the Vonage Dashboard website and under capabilities:
  3. Toggle Voice to enable this capability
  4. That hostname (the URL you just copied from Heroku) will be used followed by /answer on the Answer URL. Make sure HTTP GET is selected
  5. Do the same for /event on the Event URL. Make sure HTTP POST is selected
  6. Click on Save Changes

Improving the Dialogflow Agent

Follow the steps

  1. Open the dialogflow console and make sure you’re in the correct agent for this tutorial
  2. Click on the Default Welcome Intent
  3. Scroll down, erase the responses and add: Welcome to our Demonstration Restaurant. When and for whom would you like to book a table?
  4. Click Save
  5. Click on the Intents menu item
  6. Click on the New Intent button
  7. Give it the name of Table Booker
  8. Under Training Phrases add the following training phrases
  • A table for Nicole on the 10th at 13:00, please.
  • Could I book a slot at 14:00 on 10/12/2021 for Joseph?
  • I’d like to book a table on the 10/09/2020 at 16:00 for Amanda You will notice that as you type these sentences, some words will be outlined in a colorful highlight, make sure to double click the words and make sure they are using the correct entities @sys.person, @sys.date and @sys.time.
  1. Under Actions and Parameters, tick required for person, date and time
  2. Add prompts for each one of the parameters because in case they haven’t said that specific piece of information during the conversation, they’ll be prompted to add them
  1. In the response, you can add the following: Table booked for $person at $time on $date. Thank you!
  2. Toggle Set this intent as end of conversation, below responses
  3. Click Save

Interact Via Voice With the Dialogflow Agent

Follow the steps

  1. Call the phone number linked to your Voice API application and interact with the Dialogflow Agent
  2. Here’s a potential way you could test the conversation:
  • Vonage Websocket: Connecting your call, please wait.
  • Bot: Welcome to our Demonstration Restaurant. When and for whom would you like to book a table?
  • You: I’d like to book a table at 10:00 am on 01/09/2022 for Tanya
  • Bot response: Table booked for Tanya at 10:00 on 01/09/2022. Thank you!
  • End of call

Congratulations

Resources

--

--

--

Developer content from the team at Vonage, including posts on our Java, Node.js, Python, DotNet, Ruby and Go SDKs

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

An Introduction: Testing Apps on Android

Python Class_2

Creating a DynamoDB Stack using AWS Cloud Development Kit for Java with Gradle

AMA REWIND: ArcadeNetwork Community AMA Transcript

Simplifying Media Innovation at Netflix with Archer

Delightful Figures in Python Series: Donut Plot

I played Clawee and won a real prize! Check it out — ->!

How did I become a coderholic?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Vonage Dev

Vonage Dev

Developer content from the team at Vonage, including posts on our Java, Node.js, Python, DotNet, Ruby and Go SDKs

More from Medium

Git Feature — A new methodology to manage your development work

Improving your Github repos presentation using projects feature

How to use Tin to quickly set up Apache APISIX in minutes

OSD600 — Lab 9: Continuous Integration with GitHub Actions