More interesting questions from customers that are much more thought-provoking than you expected
Original post by Daniel Toczala on Medium
In my role as a Customer Success Manager, I deal with IBM customers all of the time. Some of my customers understand the general concept of “the Cloud”, and some have more difficulty with it. Some of them understand the basics of AI, and some struggle more with it. Most people just need to hear it explained to them in English (rather than tech-speak), slowly and patiently, and they understand. Most of the confusion is because the people who should know better (the technical people building these things), all use slightly different terminology and examples to explain the concepts. Here are some of the basic Watson questions that I have found myself answering a lot this year — and some real-world answers for them.
What Version of Watson Am I Using?
This is actually a pretty good question. Just because you are using Cloud-based services, and someone else is in charge of the upgrades, doesn’t mean that you don’t need to be aware of what version of your Watson service you are using. The Watson Assistant will require a version in your API call (or in the SDK). So does the Watson Discovery service and the Natural Language Understanding service. If you are using the SDK and writing your own orchestration layer, it’s usually a good idea to just have a constant in your code that is used by any/all API calls. Then you can make a single change (of the constant) when you change versions.
It’s important to know these things because new versions of the service could change how things work in your application. So you will want to manage how and when you move to new versions of the service — and make sure that you test your application well before upgrading to a new version of the service and releasing the changes to your end-users. Manage these service versions in the same way that you would manage versions of open-source packages that you may be using.
Is V2 of the Watson Assistant API Twice as Good as V1?
Not really. Why is there a V2 version of the Watson Assistant API? There are two versions of the Watson Assistant API because there are two different ways that people conceptually interact with the Watson Assistant service. Some people and orchestration layers keep track of user information themselves and just make static and stateless calls to the various Watson Assistant endpoints. They may or may not keep some of the context from the call, storing it in a variable, and then passing it along with their subsequent API calls. These are the users of the Watson Assistant V1 API. The V1 API is sometimes referred to as the “stateless” Assistant API. Take a look at the various endpoints — they are all clustered around major functionality areas — intents, entities, logs, messages, etc.
Some of our users and orchestration engines interact with the service on a session-by-session basis. Each session is an object, with that object has various states and properties. These users will often use the Watson Assistant V2 API, or the “stateful” Assistant API. These API calls have three major categories — sessions, messages, and logs. You will need to create a session for each user interacting with your chatbot, and you can then respond to the messages sent in by the user. The session object will retain all of the state data for you.
One of the secrets of power users of the platform is the knowledge that you can use both of these APIs at the same time, with the same Watson Assistant instance. So use the API calls (and API version) that make sense for what you are doing. Just keep in mind that the two APIs look at the world a bit differently.
Why Is Watson Disambiguating My User Statements?
One of the more powerful capabilities that Watson Assistant has is the automatic process of disambiguation when the intended user intent is not clear enough to the Assistant. You should have seen this happen if you are using the standard Watson Assistant interface — and not providing your own orchestration engine and UI. Your user makes an utterance like, “Where do I find the last train”, and Watson returns a dialog asking the user which answer is most appropriate for them, “something about train schedules, or something about train locations?”. Your Assistant was unable to discern if the word ‘last’ in the user question was a time-based or a geography-based reference. So it asks the user which one they meant.
So what triggers this behavior in Watson Assistant? There is not a definite cutoff — it depends on the relative confidence levels. If you turn on auto-learning, the model will actually learn over time based on how these disambiguation scenarios play out (which intent does the user choose?).
The disambiguation level is a sliding scale, which is within 55% of the top confidence level. So for example, if the top intent has a 60% confidence level, then disambiguation will be triggered if another intent of 33% or better (.55 x 60% = 33%) is encountered. For a situation with a top[ intent having an 80% confidence level, disambiguation gets triggered for any intent with a confidence level of 44% (.55 x 80%) or higher.
This factor may get changed or tuned in the future — although we have not changed it at all in recent memory. If you are managing the orchestration layer, you have the ability to override this and provide whatever disambiguation algorithm you want.
What is All of the Excitement About API Endpoints?
One of the main areas of fear in the minds of many of my customers is around the changing of the Watson API endpoints. Some email updates have gone out to account owners informing them of their need to move their API endpoints. What does this all mean?
IBM has moved the API endpoints for the Watson services on the IBM Cloud, making them a bit easier to use from multiple different data centers. Services that used to be based on gateway-wdc.watsonplatform.net now are based on something more like api.us-east.assistant.watson.cloud.ibm.com . The old endpoints are being phased out, and the watsonplatform.net endpoint will no longer be supported beginning on May 26, 2021.
In the past, most customers created their own orchestration layer for their chatbots and other AI based services. They would use one of the Watson SDKs (Node, Python, Java, or others) to interact with their Watson services deployed out on the IBM Cloud. These SDKs would then allow you to create sessions that communicated with your Watson services via their APIs. When creating these sessions, you have the ability to set the API endpoint being used. If no endpoint is set, a default endpoint is chosen. More recent versions of the SDK have updated the default API endpoint being used.
Some older software just depended on the default setting, which was often something like the API endpoints in the US South datacenter — API endpoints that looked like this:
https://gateway-wdc.watsonplatform.net/assistant/api/v1/workspaces/{wkspc_id}/message
Now the SDKs have updated these and the API endpoints look like this:
https://api.us-east.assistant.watson.cloud.ibm.com/instances/{inst_id}/v1/workspaces/{wkspc_id}/message
The good news is this: if you have been setting your own API endpoints, you just need to change the values that you are using to use to the new endpoints. It is considered a best practice to set your API endpoints in a constant which is set as a configuration variable (which you would be able to set in a YAML file in a Kubernetes configuration). If you are using the Watson Assistant Web Client, and the pre-built Assistant UI, then these new endpoints are already taken care of for you.
Finding what code is currently using the old endpoints is pretty simple. From an organizational perspective, you may want to follow these steps:
- Send an email out to all of the development teams that you know of, and tell them about the change in API endpoints, and let them know that the old endpoints will cease working on May 26th.
- In parallel, check the code associated with EVERY PRODUCTION implementation, and make sure that they are not using the older API endpoints. Just grep the code repository looking for
watsonplatform.net. - Next, check the mission-critical implementations, and make sure that they are not using the older API endpoints.
- Wait until May 26th. At that point, the services will stop working for some team — and that team will complain. When they do complain, ask them why they didn’t read the mail or respond to step 1 above.
Dan Toczala's Blog 