IBM Watson Fueled Chatbots - For Our Health

This has also been published at Medium.com.

With Covid-19 and all of the changes in our world, it’s interesting to think about how this impacts businesses all over the world. What can a chatbot do that is relevant in our new situation — with stay-at-home orders, social distancing, and the closing of non-essential businesses? Chatbots can do something that most local organizations need — communicate to your customers and users with a limited amount of staff, on a 24×7 basis. How many businesses are unable to staff call centers — wait times on many “help lines” can be up to two hours long. Chatbots enable you to “talk” with your customers and give them the information that they want, when they want it. Coupling your chatbot with some speech services can give you a solution that will work for your call center. A recent study by Forrester Consulting indicated that companies saw a benefit of over $5 per contained conversation handled by Watson Assistant. So what are you waiting for?

The Objections

A lot of companies and organizations would consider chatbots, and other AI technologies, but they feel like those technologies are too expensive. They might feel like they don’t have the expertise to build a chatbot on their own. They might have been right — in the past. This week, IBM announced it’s “Citizen Chatbot” campaign. This is targeted at government, healthcare, and other public agencies, and it provides for 90 days of FREE access to a fully functional chatbot with Covid19 information and guidance that is kept up-to-date with content from the CDC. It allows you to quickly and cheaply try chatbot technology to see how it can work for you.

That sounds wonderful, but a lot of my readers are not in those industries. Even though they are not in those industries, they still have a need to share Covid-19 information with their employees, business partners, and customers. Some of this information might be specific to the products and services that they provide. So if I can’t qualify for the “Citizen Chatbot” campaign, is there a way that I can do this for myself? How much effort would it be? How much would it cost?

The Simple Chatbot Solution

The most simple chatbot has three components: Watson Assistant, Watson Discovery, and your web page or mobile application. 

Simplified chatbot architecture

Your Web page or mobile application will be the user interface for your chatbot. It is what the user interacts with. It handles login (if needed), sets the colors and presentation of your application, what the forms look like, and all of the things that your end-user sees.

The Watson Assistant piece will drive the “conversation” with the user. It determines what the user wants, and then provides the needed information or guidance. It handles the more simple questions, the frequently asked questions (FAQ’s) and the general questions that your end users may have.

When questions get too specific for the Watson Assistant to handle, they are passed off to the Watson Discovery service. The Discovery service will read (or ingest) a series of documents or web pages, and will then search through this knowledge base (commonly called a corpus) for answers to those more complex or complicated questions. These are often referred to as “long-tail” questions.

Mobile applications may use the Watson Speech services to provide voice inputs and outputs to your Watson Assistant service. These additional services would incur an additional cost when deployed, but would allow your chatbot to relieve some call center volume (which could be a huge cost savings for you). These are not necessary for a chatbot deployment, but represent the ability to add some capability to your chatbot.

The Simple Chatbot Costs

The costs for any solution can be difficult to fully quantify. What will the software development costs be? How much maintenance will need to be done? How many people have to be on call to support the solution? I don’t know your labor and development costs, so I leave the calculation of those costs to you. What I DO know, is the cost of the Watson services that a chatbot would use. So let’s look at our simple chatbot.

• We use a single Watson PLUS instance. This single instance is going to cost $120 for each 1000 unique users every month. So if our bot is servicing 10,000 users in it’s first month, with some of them calling in multiple times, my monthly cost for the Watson Assistant service is going to be $1200.
• We use a single Watson Discovery instance. We were careful when creating our Discovery instance, and created an x-small size instance, since our knowledge base is less than 50k documents. This size Discovery service will cost us $500/month.
• We will use a single Speech-to-Text instance. These instances get billed based on usage — at a rate of $0.02 per minute. If your usage gets really high, this price per minute will drop (see the Speech-to-Text catalog page for service pricing). If we assume 5 minutes of speech per call (which is a loooong call…), we would have a bill of about $1000 for the month.
• We will use a single Text-to-Speech instance. These instances get billed based on usage — at a rate of $0.02 per thousand characters. If your usage gets really high, this price per thousand characters will drop(see the Text-to-Speech catalog page for pricing info). If we assume 4000 characters of speech per call (which is a lot of talking in each call…), we would have a bill of about $800 for the month.
• Note: IBM currently provides a Watson Assistant for Voice Interaction (WAVI) bundle of Watson Assistant (WA), Watson Discovery (WDS), Speech-to-Text (STT), Text-to-Speech (TTS), and Voice Gateway (VGW) for only $650 per month!! It provides 10K minutes of call time and 10K unique monthly web users.

With all of these costs added together, we see a total monthly cost of $3500 per month (Assistant’s $1200 added to $500 for Discovery, and $1000 and $800 for the Speech services) for handling these 10000 unique customers. That sounds like a lot of money, and it is. But now let’s look at the benefits that we will see.

I am going to make some assumptions, and use some “typical” numbers. I encourage you to do the math shown here, using your own numbers and costs, and see what your particular situation is. For this example, I will assume that we have a local call center that has been struggling to handle those 10,000 calls a month. We’ll assume that our chatbot can deflect only half of those calls (most customers report far more significant call deflection rates). So we figure that 5,000 calls a month are deflected. Let’s take that $5 per deflected conversation benefit from earlier, and we can cut it in half — to $2.50 — let’s be conservative in our benefit estimates. Multiplying those, we end up with a benefit of $12,500. Keep in mind that we are trying to be conservative with our benefits calculations. Now all of sudden our $3500 a month in costs doesn’t look so bad — in fact we have a net benefit of $9000. If we ramp up with more users per month, the costs, and benefits, increase on a fairly linear scale.

Note: Simple math with the WAVI bundle mentioned above also results in a fairly linear cost/benefits curve, with only $650 in costs against a benefit of $12,500 for those first 10000 users.

What Happens Now?

Many of you will be wanting to start building out your chatbot, but your organization will want to manage the effort and will want some sort of project plan. So let’s give them one — and begin to set expectations.

Sample Chatbot Project Plan

Looking at this simple chatbot project plan, you can see the high-level tasks involved in rolling out a chatbot. Provisioning resources is simple, and you can do all of your provisioning in an hour or less. Keep in mind that you should do this within a functional IBM Cloud account (see my post on Getting Started Right With the IBM Cloud). 

You then will need to do some work on your dialog skill, first importing some pre-built intents, training data, and dialog flows. This is simple and should take less than an hour. Updating these to fit your particular situation will take a couple of days.

In parallel to your dialog work, you can create a search skill, and implement and execute the web crawling needed to populate your knowledge base for handling those “long tail” questions. All of these tasks can be done during the first few days of the project.

At this point you will need your existing development and operations teams to work together in getting the code to deploy your chatbot widget added to your homepage (or wherever you decide to host your chatbot). I strongly suggest doing this in some kind of staging or test environment. You can then test in this environment, and eventually deploy these changes to your production environments. All in less than a week.

After this initial deployment, you will have some longer workstreams to execute on, as well as some recurring work. For the recurring work, you’ll need to set up some iterative process for reviewing your chatbot results, and identifying improvements that need to be made. You will need to identify areas where things are changing, and where your chatbot needs to change.

The two longer workstreams that are needed to be done involve the automation of the testing process and the automation of the promotion/deployment processes. These are necessary if you want to make your chatbot a smoothly functioning and updating application. The processes and tools that you use here should be able to be used for ANY chatbot that your organization creates.

Let’s Get Ready for Building!

Before I launch into an overview of the steps that you need to take to build your very own Covid-19 Chatbot, you should be aware that there are now hundreds of “How to build your own Covid-19 chatbot” articles out there. Many of them are really good, and you might prefer their approach to what I have here. So I’ll list a few of them here — take a look at them and use the approach and the instructions that seem most straightforward to you.

• Create a crisis communication chatbot and connect it to news and COVID-19 data sources— this “Call for Code” effort is pretty good, one of my good friends and co-workers was one of the driving authors behind this effort. Includes a news update for long-tail questions. We use their intents, entities and dialog nodes in this example.
• Simple Covid-19 Chatbot — a great video that shows you how to do this in 4 minutes. He moves fast, so you’ll probably have to start/stop the video quite a bit, but it is simple and straightforward.

The first thing to do is to get a “functional account” in your domain. We touched on this earlier (my post on Getting Started Right With the IBM Cloud), but it is important. Make sure that your “owning” account is not tied to a single person — but is more of a service account that multiple people can administer.

Using that “functional account”, get yourself an IBM Cloud account. Some organizations may already have an IBM Cloud account, in which case, you’ll just need to make proper arrangements with the account owner.

We need to do some basic housekeeping and setup of our Cloud account, so we can more easily manage access to resources. We’re going to want to create a new Resource Group and some Access Groups for our chatbot project. So first go to “Manage -> Account”, and then click on “Resource Groups”. You’ll see a list of resource groups (maybe just a default resource group — it depends on if this is a new or existing account). On this screen, we’re going to click the blue “Create” button to create a new Resource Group. 

Resource Group creation

Call your new Resource Group something informative, like “Covid_Chatbot_RG”. Make sure to add the “_RG” at the end of your Resource Group name — it’s never to early to start using naming conventions.

Now we want to create some Access Groups — so we can easily add users to our project based on the role that they will play. So select “Manage -> Access (IAM)”, and then click on the “Access Groups” option. Now we’ll create three access groups, and name them with our resource group name, followed by the role, and then an “_AG” (which shows they are an access group). So for my three access groups, I created the following: “Covid_Chatbot_Admins_AG”, “Covid_Chatbot_Developers_AG”, and “Covid_Chatbot_Users_AG”. After creating these access groups, you should now see them on the “Access Groups” page.

The last thing to do with access groups is to set permissions for the access group. From the “Access Groups” page, click on the “Covid_Chatbot_Admins_AG” group (or your equivalent). Once the access group comes up, click on the “Access Policies” tab, and click the blue “Assign Access” button. This screen is where we will assign access for each of the Access Groups. Here is what you need to set for each group:

For the “Covid_Chatbot_Admins_AG” group:

• Select “IAM Services”, “All Identity and Access enabled services”, “Resource group: Covid_Chatbot_RG”, and “All regions”. Check every box for platform access (Viewer, Operator, Editor, Administrator), service access (Reader, Writer, Manager), and resource group access (Viewer, Operator, Editor, Administrator). Note that you may need to click on the “View All” link to see all of your options for resource group access. Press the “Add” button at the bottom of the window.
• Select “Account Management”, and “All Account Management services”. Check only two boxes for platform access (Viewer, Operator), and press the “Add” button at the bottom of the window.
• Finally, press the blue “Assign” button in the right-hand pane, to assign these permissions to this access group.

For the “Covid_Chatbot_Developers_AG” group:

• Select “IAM Services”, “All Identity and Access enabled services”, “Resource group: Covid_Chatbot_RG”, and “All regions”. Check two boxes for platform access (Viewer, Operator), and three boxes for service access (Reader, Writer, Manager). Press the “Add” button at the bottom of the window.
• Select “Account Management”, and “Support Center”. Check three boxes for platform access (Viewer, Operator, Editor), and press the “Add” button at the bottom of the window.
• Finally, press the blue “Assign” button in the right-hand pane, to assign these permissions to this access group.

For the “Covid_Chatbot_Users_AG” group:

• Select “IAM Services”, “All Identity and Access enabled services”, “Resource group: Covid_Chatbot_RG”, and “All regions”. Check one box for platform access (Viewer), and one box for service access (Reader). Press the “Add” button at the bottom of the window.
• Select “Account Management”, and “Support Center”. Check three boxes for platform access (Viewer, Operator, Editor), and press the “Add” button at the bottom of the window.
• Finally, press the blue “Assign” button in the right-hand pane, to assign these permissions to this access group.

The last thing to do with access groups is to set your account into the admin access group. From the “Access Groups” page, click on the “Covid_Chatbot_Admins_AG” group (or your equivalent). Once the access group comes up, click on the “Users” tab, and click the blue “Add users” button. Now select your account, and then click the blue “Add to group” button.

Now we want to people invited to our project area. Select “Manage -> Access (IAM)”, and then click on the “Users” option. Now you can add the email addresses (IBM Cloud accounts) of the people you want to invite to your effort by clicking on the blue “Invite users” button. Add your users in groups, separated by commas, and assign each group of email addresses to a particular Access Group by clicking the blue “Add” link for the access group that you want assigned. This will give our new users access to the areas of the account that they need access to.

Now we’ll log in using your own account (in my case that might be tox@ibm.com), so we can check out these changes and get started with our chatbot.

Getting Services Created

The first steps in this project involve getting your Watson services deployed. We’re going to try to keep this as simple as possible — if you are looking for more complex deployment examples, see some of the links in the previous section, or reach out to me and I can probably point you in productive directions.

Now we want to make sure that we create these in the right place, so log into the IBM Cloud in your own account, and switch your context to be operating in the newly created account (where the chatbot is going to “live”). To make this context switch, go to your top bar, and look for your current context. Click on it and then choose the more general account (and not your personal account).

Switching your IBM Cloud context

Now let’s start creating some services. Click on the “Catalog” menu option, and then look at the AI offerings. You will want to choose the Watson Assistant. When you click on it, it will give you some options for creating your service. You will want to choose a geographic region (or data center) where you want to create your instance. Choose one that is relatively close to your physical location. Then select a plan (choose the Plus plan). Now you will need to select a service name (I have a naming convention, so mine is called “WA_Covid_Chatbot_Tox”) and a resource group (we’ll use the one we created, “Covid_Chatbot_RG”). Once this is all set, click the blue “Create” button in the right-hand pane to create your Watson Assistant instance.

While this instance provisions, let’s go and create the service that will handle our long-tail questions. Click on the “Catalog” menu option, and then look at the AI offerings. You will want to choose the Discovery service. When you click on it, it will give you some options for creating your service. You will want to choose a geographic region (or data center) where you want to create your instance. Choose the same one as before. Then select a plan (choose the Advanced plan). Now you will need to select a service name (I have a naming convention, so mine is called “WD_Covid_Chatbot_Tox”) and a resource group (we’ll use the one we created, “Covid_Chatbot_RG”). Once this is all set, click the blue “Create” button in the right-hand pane to create your Watson Discovery instance.

Begin Doing Some Assistant Work

So now we are ready to do some damage. Let’s go to our IBM Cloud dashboard, and take a look at our services/resources. Under the services section, you should see the two services that you created. Because you were smart and you used naming conventions, you know by the names what services these are. So you click on the name beginning with “WA” to open your Watson Assistant instance. Once it opens, click on the blue “Launch Watson Assistant” button. You’re now into the Watson Assistant tooling.

You now see a page with your Assistant displayed, it’s called “My first assistant”. You can see that it has one skill, under Dialog, called “My first skill”. Click on the second icon (called “Skills”) in the left-hand nav bar to pull up your list of skils. You should now see a list of all of your skills (you only have one right now). Click on the three dots in the upper right-hand corner of the skill, we need to rename this thing. Select “Rename”, and then rename your skill to “My Covid Skill” or something similar. Now just click on the main area of the box — and you are now in the skill tooling.

Starting off with a new skill

We’re starting from scratch, but we want to get this deployed quickly. So lets grab some work and data that has already be done for us. Click on the three dots in the upper right of the display, and select the “Import” option. We are going to import some JSON that represents a pre-built generic Covid-19 chatbot. You will want to use the fileCovid Json as the import file.

Note: You need to download this file to your local machine to make this work. If you’re not a GitHub person, then just click on the “Raw” button, and a new page will be displayed in your browser window. Once this is displayed, in your browser, just save the page by clicking “File -> Save Page as…”, and save it in JSON format. This content is from the “Call for Code” competition.

You can now poke around and take a look at the Intents that were added to your skill. Also take a look at the Entities, and the Dialog nodes. Don’t change anything — but look around at things. Take some time, after the import of new data your Watson Assistant needs to take some time to learn (called training) all of the new things that you just threw at it. After a couple of minutes, you can click the “Try it” button, and you should see a pane open up on the right-hand side of your browser. This is where you can test and check out your chatbot. Make sure that there is no flag that says that Watson is still training — and then try typing in some questions for your chatbot.

After that, you can now take a look at some of those dialog nodes. Click on one of them — and see how the responses are returned to the end user. Those conditions, the ones with the hashtag in front, are based on your intents. Those conditions with the @ in front of them are based on your entities. Using the detected intents and entities, we determine what the end user is asking about, and we supply an appropriate answer to them.

Let’s Look at the Long Tail

Now we need to configure our other service, the Watson Discovery service. Let’s go to our IBM Cloud dashboard, and take a look at our services/resources. Once again you smile a little, you were smart and you used naming conventions, so you click on the name beginning with “WD” to open your Watson Discovery instance. Once it opens, click on the blue “Launch Watson Discovery” button. You’re now going to see the Watson Discovery tooling.

The first thing that we need to do is to create our Discovery environment and choose a size for our Discovery instance (for more details on this, check out my blog post on Watson Discovery at the Size You Want). So in your Discovery window, find the environment icon (in the upper right corner), click on it, and then click on the blue “Create environment” button. The size of your environment is important — it determines the cost of your Discovery service. We wanted a quick chatbot, and we don’t anticipate a huge amount of documents needing to be stored, so we choose the “X-small” option, under the development tab. Once you find this option, click on the blue “Set up” button to begin setting up your environment, which will take a few minutes.

Once your environment is all set up, we’ll set up a simple collection and a simple web crawl to populate that document collection. You’ll notice that there is a button to “Connect a data source”. Press it. Next we’ll select “Web Crawl” as our data source. Now we get to add URL’s of the sites that we want to crawl for information. Let’s begin with these sites (you may want to add your own information sources):

• https://faq.coronavirus.gov/ (1 hop — sync once a day)
• https://www.cdc.gov/coronavirus/2019-ncov/faq.html (1 hop — sync once a day)

As you add these URL’s, click the small gear icon that comes up with them as they are added. Then configure the number of hops for each data source. The web crawl will follow links, and links from those pages, and this is the depth of that link following that will be used. Once you have added your sources, click on the blue “Save & Synch” button. This will start your initial web crawl of data.

Your web crawl has now begun. On this screen, you can see the name of your new collection, which is just some random collection of characters. We want to change that. We’re going to call our new collection “CDC_Covid_info”, so click the small pencil icon next to that random name, and change it. So now we’ll just wait for those sites to get crawled and processed. This takes a little while, so we’ll just continue on and let it continue in parallel.

Note: Don’t panic if/when your Discovery web crawl begins to display numbers of documents that have failed. Documents fail to get ingested for a number of different reasons. Usually we just cast a wide net, and if some pages don’t get properly picked up, then we’ll find that information somewhere else.

Bringing it Together

Check your watch — how long has it been since we started? All we need to do now is connect these services, and then deploy our chatbot. Let’s start by connecting the services.

Go back into your Watson Assistant instance, and let’s look at the Assistants, which you can see by clicking on the top icon in the left-hand navigation pane. You can now see your only assistant — click on the rectangle to go into the assistant. Once inside, you can see your recently imported dialog skill. If you click on that box, you go to that familiar tool where you can modify your intents, entities, and dialog nodes. Let’s not do that right now, instead let’s look BELOW our dialog skill. There is a space there for a Search Skill. Let’s go and add one of those!

So press the blue “Add search skill” button, and you will be asked to name your search skill. Let’s name it the same thing as our document collection in Watson Discovery (“CDC_Covid_info”)— that way we won’t be confused or misled later — and hit the blue “Continue” button. On the next screen we need to choose our Watson Discovery instance (“WD_Covid_Chatbot_Tox”), and our web crawler collection of documents (“CDC_Covid_info”). Then press the blue “Configure” button, which will bring you to the last configuration screen. This screen allows you to tailor the information returned to your user. For now, we’ll just stick to the default values. So the last thing to do is to press the blue “Create” button to establish the collection as your source for answers to any “long tail” questions that your chatbot gets.

Now we need to make sure that we can get there. So let’s go into the Dialog skill for our chatbot. Scroll all of the way to the bottom, and find the dialog node called “Anything else”. Click on this node and open it up. If you look at it, it basically states that if it sees anything not covered in your intents, it will fire off a webhook. We don’t have time to do webhooks today (see some of the great materials on webhooks by Mitch Mason, Morgan Langlais, or this simple webhooks demo), so we’ll get rid of this. Click on the minus sign next to all but one of your options under the “Assistant Responds” section. For the last remaining option, click on the small gear, and configure the response. Under the “If assistant recognizes”, the value should be “anything_else”. Under the “Assistant Responds” area, select “Search skill”. Once you’ve done this (which tells the chatbot if it cannot detect an intent, to send the question to the search skill — or Discovery collection), press the blue “Save” button.

Testing and Releasing

Your chatbot is done — it might not feel that way, but it is. So let’s check it out. First we’ll want to do a basic check out of the answers to common user questions. So go back into your Watson Assistant instance, and let’s look at the Assistants. You know how to get there by now — if you don’t, then scroll back up and figure it out again.

Click on your assistant, and in the upper right of the window, you will see a section for integrations. In that section is a link that says “Add integration +”. Click on that link.

You have a few options here, but right now we’re just going to use the “Preview link” option. This is there so you can test out your chatbot with a private link, before you have to expose it to the world. So click that option, and just leave the default name for your preview link, and press the blue “Create” button. Once you do — a link will be generated that will allow you to see your chatbot in a web browser. Click on that link and ask some questions, check out how your chatbot is working.

When you are ready — and you should be ready pretty quickly — it’s time to deploy this chatbot. Let’s go back to the assistant page, and click on that link that says “Add integration +” again. This time we’ll add a “Web chat” integration. So click that option, and again we will just leave the default name for your name, and press the blue “Create” button. Now we see some options to customize your UI, and as you change these options you can see a sample window to the right of what your chatbot will look like. Once you are done playing with the title bar name and the colors that you want, you can just copy the code in the following section, and have it added to the web page where you want your chatbot. Simple as that.

Now What?

Now you need to update this chatbot. You should learn more about how Intents work, how Entities can be used to help make your chatbot responses more targeted, and how Dialog nodes work. I like the Watson Assistant landing page (full disclosure — I help curate this page) for links to more learning resources. You can also take some of the online courses offered if you prefer to learn that way. The BEST way to learn is by doing. Make changes to your chatbot and see how it reacts. Don’t be scared — if you ever want to try something out, just duplicate your skill first, and then work on the copy. Then you always have the original to go back to if you really screw something up.

You will want to make sure that your responses haven’t needed to change due to changing conditions, and you may want to add new intents for new situations that your users might ask about.

If you’re really stuck, or just overwhelmed with the possibilities, you can always reach out to me. I can help you talk through your options, and I can give you some good advice to move forward with. Just reach out to me at dtoczala@us.ibm.com.