Authentication for your App on the IBM Cloud

I like to answer customer questions on the IBM Cloud, and about using Watson services on the IBM Cloud. Sometimes I know the answer… sometimes I have to go and find the answer… and sometimes I create the answer. As I encounter interesting questions, I like to share my solutions and answers on this blog – so that way I am able to help other people who may be encountering the same issues.

Recently I have been seeing a lot of questions about how authentication on the IBM Cloud works, and what the “best practices” around authentication are. It’s a large topic, and I cannot cover it completely in this blog post, but hopefully, I can cover it to a depth that will help you make decisions about how you want to structure your IBM Cloud/Watson application.

Getting Into The IBM Cloud

You get into the IBM Cloud using your own username/password to authenticate. You might even use two-factor authentication to get in. The access to all of the services and accounts on the IBM Cloud is controlled by the IBM Cloud Identity and Access Management (IAM) service. This service validates your identity and then assigns your access and permissions based on that identity.

When an application attempts to access resources on the IBM Cloud, it too needs to have it’s identity validated. Most IBM Cloud users will create either a service account (with its own login, password, and credentials) or will create some specific service credentials for every cloud service.

So the first step is to have your application authenticate itself, with some sort of credentials. To do this you will make your API call to the API endpoint that you are attempting to access. This call will go through the IBM firewall, which takes care of traffic routing, ingress, load balancing, and some other “big picture” types of things. Your request is then routed to the authentication service. This service will attempt to authenticate your request.

Basic Watson Service Request Flow

It is at this point that things get interesting for an application developer. You must choose from one of two paths:

  1. You can give the authentication mechanism an API Key. The authentication service will then validate the API Key. It does this by calling the IAM Identity service. Once you have been authenticated, your request is passed along to the appropriate Watson service instance.
  2. You can give the authentication mechanism an API Token. To get this API Token, you will first need to call the IAM Identity service, give it your API Key, and ask for an API Token. You can then make your call and give the authentication mechanism this API Token.

So why would you want to take approach #2? You want to take approach #2 as much as possible because when you authenticate via an API Token, you do not make an additional call to the IAM service when authenticating. These API Tokens are good for 60 minutes.

How Much Do I Save?

Let’s look at a simple example. If your application will make 1000 calls to the Watson service in a typical hour of usage. If you just use the API Key approach, you will make 1000 additional calls to the IAM service. Each call will the API Key will require the authentication service to make a request to the IAM service.

If you decide to use the API Token method, you will make an extra call to the IAM service to get your token, and then you will make your initial service call. This is 1 additional call to the IBM Cloud. So in a typical hour, this will save you 999 service calls within the IBM Cloud.

The savings here for a single call to a service may be difficult to measure and probably would go unnoticed by an end-user. But during periods of high traffic and high load, this reduction is traffic and load will make your application much more robust, responsive and resilient.

Why even bother with the API Key approach? It is useful for testing scenarios and for times when you just want to issue curl calls by hand, directly to the API. In these cases, it is much nicer to just do the call once, rather than having to get a token and then use the resulting token (which can be large).

How Do I Do It?

Set up a small section of code to request your API Token, and then save that token and insert it into every API call made to services on the IBM Cloud. You’ll then need to decide on how you want to handle renewing your token, since every token is only good for 60 minutes.

You can either set a timer for 50 or 55 minutes and wake up to renew your token before the previous token expires. The other option is to just handle authentication errors in your application by refreshing your token and trying to execute your request again with the new token.

Another useless IBM Cloud Billing Script …. or not

Recently I have have had a customer ask me some questions about billing on the IBM Cloud. Specifically, they wanted to be able to download their billing information. Now I know that you can do this interactively from the IBM Cloud. You just go to your usage page and hit the little Export CSV button in the upper right-hand corner. This will dump out a CSV file of your current months usage for you.

“That’s nice…..but we want to be able to break down usage and get it into a spreadsheet”, was the response from my customer. “But CSV format IS a spreadsheet”, I thought to myself. So I decided to ask some more questions. It turns out that my customer is interested in AUTOMATING the administration of their IBM Cloud. This means that they want to dump not just one month’s worth of data, but a full 12 months of data into statements that will outline internal chargebacks (based on resource groups), as well as showing visual graphs that summarize usage and spending over the past 12 months.

Now I understood why the “button in the upper right” wasn’t going to work. They wanted some sort of script that would run automatically, to generate all of this content for their IBM Cloud user community. I had wanted to do some work with the Billing and Usage API and Resource Manager API on IBM Cloud, so I decided to script something up for them.

This project also led me to using the Resource Controller API – which is slightly different from the Resource Manager API. The Resource Controller API focuses on the various different service instances (resources) that have been created within an account. The Resource Manager API deals more with the resource groups defined within the account.

Note: If you’re reading this post and serious about doing this sort of automation for your IBM Cloud, PLEASE check out the API links above. It is possible to use different API calls to do more targeted operations – instead of the very broad implementation that I have done.

As usual, I decided to work within Watson Studio, and I decided to write my script in Python. Watson Studio includes some environments that have pre-defined collections of packages – including ones you will depend on like pandas, numpy, and IBM_Watson. It also allows me to run my script up on the IBM Cloud (not on my local machine), version my script, and dump my results to IBM Cloud’s Cloud Object Storage. My goal was to show this customer how to do a dump of just the services with usage (and not EVERY service, since there are many services out there which are on Lite plans). I also wanted to highlight how to get to some of the information available via the Resource Controller API.

I quickly learned that in order to get ALL of the information that I needed, I would need to cross-reference some of the data coming back from these various IBM Cloud API’s. That meant using all of the IBM Cloud API’s mentioned above.

So now I have this Python code all sitting in my new GitHub repository called IBMCloudBillingScript. This script only really does a lot of the same things that the “Export CSV” button does – so this makes this script kind of useless. The reason I built it, and have shared it on GitHub, is because we often want to AUTOMATE things based on this information. This script shows you how to get (and manipulate) this kind of information about your IBM Cloud account.

I’m Having an Issue on IBM Cloud Part 2 – What is Happening on the IBM Cloud?

By: Daniel Toczala 

Note: This is the second blog in a series of blogs that I am co-authoring with Paula Williams, as part of an “I have an Issue…” series on the IBM Cloud. The first article, Part 1 – Why Can’t I Create Anything can be found on my WordPress site. The next article, Part 3 – Who you Gonna Call ? For Your IBM Cloud Watson AI Software As A Service (SaaS) Needs can be found on Paula’s Medium site. These blog posts will cover how to deal with common issues and roadblocks for users of the IBM Cloud. 

I like helping my IBM Cloud customers, and I like dealing with the technology.  Every new technology (and even established technologies) has a learning curve.  My goal in writing this series of articles is to help you more quickly conquer that learning curve with respect to the IBM Cloud.   

Today’s article deals with understanding what is going on in the “big picture” with the IBM Cloud.  How do you know what new services are available on the IBM Cloud?  How do I know when maintenance windows will occur?  How do I find out when services are getting deprecated and retired?  If my services are down, is it just me?  Is it other people too?  Is the whole data center down? 

Checking the Current IBM Cloud Status 

When things are not working, or seem to be slow, the first place I check is the overall IBM Cloud status page.  You can find it here ->  There are a few different ways to look at this page.  The first tab shows the Status of the overall cloud – which services might be unavailable and which regions are impacted.  There are four other tabs, and they show other information.  One is for Planned Maintenance, and this shows upcoming maintenance windows and describes their impact on users of the services.  It’s always good to check this once a week to see what upcoming activities may impact your users and cloud projects.  Another tab is for Security Bulletins, and this one shows important security bulletins and events that you will want to be aware of.  There is also a tab for more general IBM Cloud Announcements, which contains general cloud announcements and event notifications.  The final tab is for History, so you can see the events of the past few days, and see what the resolution of those events was. 

This is a lot of different tabs for you to check.  I have to admit, even as a frequent user of the IBM Cloud platform, I rarely check these tabs on a daily, or even weekly, basis.  Instead, I subscribe to an RSS feed that will give me push notifications of IBM Cloud events as they get posted.  For those of you unfamiliar with RSS, it is a publishing technology which allows users to “subscribe” to event streams.  There are a bunch of free RSS readers out there, just look one up and install it.  Then point your RSS reader at the IBM Cloud RSS feed.  The RSS link is on the IBM Cloud Status page – just click on the Subscribe icon on the right-hand side of the page. 

Signing Up For Email Notifications 

Another thing that IBM Cloud account owners and IBM Cloud Administrators should do is to sign up for email notifications.  You can have the account owner (your IBM Cloud account which “owns” the subscription) get notifications each month when certain events occur.   

Setting this up is easy, for the account owner.  Log into the IBM Cloud as the account owner, and then select Manage -> Billing and Usage from the top navigation bar for the IBM Cloud.  In the resulting screen, look at the menu on the left side of the browser, and select the Spending Notifications option. 

On this Spending Notifications screen, you should now be able to specify sending spending notifications to yourself for any of the conditions specified.  Set your limits, and be aware that you will be notified when you reach 80%, 90% and 100% of your specified threshold.  Your Spending Notification screen should look similar to this: 

Click in those checkboxes to make sure that you get emails sent to you whenever those threshold limits are hit. 

Why Can’t I See That Support Ticket? 

I like the IBM Cloud, but on occasion you will need to open a support ticket because you have run into an issue on the IBM Cloud, or with one of the IBM Cloud services.  In order to open up a support ticket, click on Support on the top menu bar.  In the resulting Support page, click on the Manage Cases tab, and you will see a list of support cases that you are involved with. 

Be aware of the fact that this Manage Cases page has a filter which will only show support cases that you are involved with, and that are in some open state.  You may want to go and change your filters, to be able to see additional support cases.  If you are not able to see a support case, it could be because your organization has not given you the ability to see or open support cases for the organization.  If this is the case, then you’ll need to ask your IBM Cloud administrator to give you that capability.  The Create A Service Support Case documentation page has a great description of the process used to create a support case.

If you are the IBM Cloud administrator, then you will need to go to the Manage –> Access (IAM) page, and then go to Access Groups.  Once there, create a new access group, and make sure that it follows your naming conventions.  A good example might be, “SupportTicketAccess_AG”.  Once the access group is created, you’ll see the Access Group page.  Click on the Access Policies tab, and then on the Assign Access button.  Now you will need to select Assign Access to Account Management Services.  Select the Support Center service, and then apply ALL levels of access (Administrator, Editor, Operator, and Viewer) to the support center.  Now all you need to do to give users access to all of the support tickets for an organization is to add them to this access group. 

Note that you could create finer-grained access groups, like “SupportTicketViewer_AG”, that would only allow limited capability with support tickets.  Just create the additional access groups, and change your assignment of levels of access accordingly. 


Now I’m getting 5438 email messages a day about things going down – are things really THAT bad??  OK – maybe 5438 is a bit of an exaggeration, but you get the idea…. 

You have subscribed to get email notifications of outages in the IBM Cloud.  Nice job – you should be proud of yourself for being proactive!  Our IBM Cloud has a lot of different customers, all co-located with services in a lot of different data centers.  When our infrastructure team detects a loss of service (let’s say a machine dies, which causes some IBM Cloud service to fail for the 5 customer instances running on that machine), they want to notify our customers as soon as possible.  So we send out an automated warning email to our users.  This is all nice automation, and allows us to be “good” Cloud providers and let our customers know when things go wrong. 

Now we get to the not-so-pretty part.  At the time this happens, we cannot tell EXACTLY which 5 customer instances have gone down, so we err on the side of over-communication, and let EVERY CUSTOMER IN THAT DATA CENTER know that they MIGHT have lost service.  We didn’t want to ignore or pretend the errors weren’t happening – so we took this approach.  Unfortunately, these things happen relatively frequently, and while they are short in duration and limited in scope (only a couple of customers lose service for a short period of time), the email blast to customers is EXTENSIVE.  Inside of IBM we half-jokingly (with accompanying eye roll) refer to this as our BLAST RADIUS. 

What does this mean for you?  It means that you will get a lot of notices, only 5-10% of which will actually apply to you.  We SHOULD watch this issue though, as this is a known (and painful) issue that IBM is currently addressing and rolling out fixes for.  As these fixes and changes to the IBM Cloud get implemented, the percentage of notices that actually apply to you will increase from 5-10% to 100% (meaning we only notify you about things that WILL actually impact you). 

I’m Having an Issue on IBM Cloud Part 1 – Why Can’t I Create Anything?

By: Daniel Toczala

Note: This is the first blog in a series of blogs that I am co-authoring with Paula Williams, as part of an “I have an Issue…” series on the IBM Cloud.  These blog posts will cover how to deal with common issues and roadblocks for users of the IBM Cloud. Check out her post on “What the Heck is a CSM?

I like helping my IBM Cloud customers, and I like dealing with the technology.  Every new technology (and even established technologies) have a learning curve.  My goal in writing this series of articles is to help you more quickly conquer that learning curve with respect to the IBM Cloud.

Typical Issues That People See

Some typical issues that users experience when working with the IBM Cloud will often be with respect to their services.  These could be one of the Watson services (like Watson Conversation or Watson Discovery), or maybe Cloud Object Storage, or a DB2 or Mongo database.  The issues that people will typically experience fall into one of two categories:

I’ve Created Something That I Cannot See

These situations have you going out and creating an instance of a service, but now you just can’t seem to figure out how to find it or how to get to it.  It’s almost as if we have decided to hide it on you.

In these situations, it is best to first figure out WHERE you are on the IBM Cloud.  Look in the upper right corner of your browser window.  As soon as you log into the IBM Cloud, you’ll see something like this:

Finding the area you are working in on the IBM Cloud

That text in that small black block tells you WHERE you are.  It tells you the account that you are operating in.  Clicking on this box you can CHANGE where you are looking.  Confused yet?  Maybe we should step back a bit….

You have an ACCOUNT and IDENTITY on the IBM Cloud.  Your IDENTITY from an IBM perspective is typically an email address, which is also your IBM ID (for our example let’s assume that mine is  It has the same username/password as your IBM ID (which you might use for things on  You might even have a Federated ID, where we use your company email address/identity and your company authentication mechanisms.  Some things to remember when looking at your IBM ID:

  • Just because you have an IBM ID, doesn’t mean that you have an IBM Cloud account.  You will need to register for an IBM Cloud account. While both the IBM Cloud and both use your IBM ID, they are DIFFERENT domains.
  • When you sign up for your IBM ID, use a valid email address as your user ID.  You will need to VALIDATE your account by responding to an email that is sent to (you guessed it!) your user ID.  So in my case I cannot sign up for an IBM ID at (unless I have convinced my corporate IT folks to give me that email address).

Now let’s get back to finding those cloud resources that we created.  Once you see what context you are in, you will have a better idea of what you can EXPECT to see.  Since I am an Acme Co. employee, I have been added as an approved user of the Acme corporate account.  What does this mean?  Well the Acme corporation created a different account, a corporate account, associated with a service account called  This account has a subscription which provides it with a set amount of “credits” for IBM Cloud services, which it burns down over time.  Since I am a member of this account, I can create IBM Cloud services in this corporate account, and their costs get assigned to the corporate account.  IBM Cloud services are billed based on where they are LOCATED (logically), and not based on who created them.

So now you hopefully have a better feel for how your account fits into the grand scheme of things, maybe you can find out WHERE that Watson service that you created is located, by looking at the various different contexts that you operate in.

Advanced Developer Note: Your IBM ID is based on an email ID.  So I have an IBM ID for, but I also have one associated with my personal email address ( I use my account for doing my regular work, and I use my personal email based ID to do open source work.  That account is a trial account (or maybe I even attach my personal credit card to it), and I am careful not to rack up big charges on the account.  I use it for doing simple little things in the cloud environment.

I Cannot Work With Something I Created

These situations are a little different.  You are able to create some service, but you are then unable to access it.  Either the service is broken, or down, or just not responding to your repeated attempts to use it.  Or maybe you can see a service but you just cannot create it.

So let’s go back to that account.  First of all, you need to check and make sure that the account that you are attempting to create a service in is able to pay for that service.  If the account has a credit card associated with it, which guarantees payment for cloud services used, then the account is referred to as a “PAYGO” (short for pay-as-you-go) account.  People who use things like GitHub and other SaaS based services should be familiar with this model.  If the account has prepaid for services, via an IBM Cloud subscription, then it is referred to as a “SUBSCRIPTION” account.  Either “PAYGO” or “SUBSCRIPTION” accounts can create any type of service.  Your personal account might not have a guaranteed payment method, like my account.  In that case you have a “TRIAL” account.  “TRIAL” accounts can create lite (or no charge) instances of most services on the IBM Cloud.  TRIAL accounts will not be able to create more robust versions of those services until they either become “PAYGO” or “SUBSCRIPTION” accounts.

So let’s get back to our example.  My account is a “TRIAL” account (I’m not paying for anything!), but since I am an approved user of the account (which is a “SUBSCRIPTION” account), I can create non-lite service instances within THAT account.  There is one hitch…. I have to have PERMISSIONS granted to me to be able to see particular logical areas of the Acme IBM_Cloud_Admin account.

What Are These “Logical” Areas?

There are two different types of logical areas where IBM Cloud resources can be created.  Each is based on a different security model. 

The Cloud Foundry security model uses the concept of Organizations (called “ORGs”) and Spaces.  These “Orgs” and “Spaces” live in a hierarchal model, with a single Org hosting one or more Spaces.  The administrator/owner of an IBM Cloud account will create these Orgs and Spaces, and will assign people various roles in each org/space.  These roles determine what a user can do within that particular logical environment.  You need to make sure that your account has access to the orgs and spaces that you need to work in.

The IBM Access Management (IAM) security model is based on Resource Groups.  Each resource group may have a series of Access Groups associated with it, and these Access Groups can be used to provide fine-grained access controls and role management.  You need to make sure that your account is enabled to do what is needed for the resource groups that you need to work in.

You can learn all about Orgs, Spaces, Resource Groups, Access Groups and best practices for organizing your IBM Cloud account by reading my blog post entitled, Getting Started Right on the IBM Cloud.