I like to answer customer questions on the IBM Cloud, and about using Watson services on the IBM Cloud. Sometimes I know the answer… sometimes I have to go and find the answer… and sometimes I create the answer. As I encounter interesting questions, I like to share my solutions and answers on this blog – so that way I am able to help other people who may be encountering the same issues.
Recently I have been seeing a lot of questions about how authentication on the IBM Cloud works, and what the “best practices” around authentication are. It’s a large topic, and I cannot cover it completely in this blog post, but hopefully, I can cover it to a depth that will help you make decisions about how you want to structure your IBM Cloud/Watson application.
When an application attempts to access resources on the IBM Cloud, it too needs to have it’s identity validated. Most IBM Cloud users will create either a service account (with its own login, password, and credentials) or will create some specific service credentials for every cloud service.
So the first step is to have your application authenticate itself, with some sort of credentials. To do this you will make your API call to the API endpoint that you are attempting to access. This call will go through the IBM firewall, which takes care of traffic routing, ingress, load balancing, and some other “big picture” types of things. Your request is then routed to the authentication service. This service will attempt to authenticate your request.
It is at this point that things get interesting for an application developer. You must choose from one of two paths:
You can give the authentication mechanism an API Key. The authentication service will then validate the API Key. It does this by calling the IAM Identity service. Once you have been authenticated, your request is passed along to the appropriate Watson service instance.
You can give the authentication mechanism an API Token. To get this API Token, you will first need to call the IAM Identity service, give it your API Key, and ask for an API Token. You can then make your call and give the authentication mechanism this API Token.
So why would you want to take approach #2? You want to take approach #2 as much as possible because when you authenticate via an API Token, you do not make an additional call to the IAM service when authenticating. These API Tokens are good for 60 minutes.
How Much Do I Save?
Let’s look at a simple example. If your application will make 1000 calls to the Watson service in a typical hour of usage. If you just use the API Key approach, you will make 1000 additional calls to the IAM service. Each call will the API Key will require the authentication service to make a request to the IAM service.
If you decide to use the API Token method, you will make an extra call to the IAM service to get your token, and then you will make your initial service call. This is 1 additional call to the IBM Cloud. So in a typical hour, this will save you 999 service calls within the IBM Cloud.
The savings here for a single call to a service may be difficult to measure and probably would go unnoticed by an end-user. But during periods of high traffic and high load, this reduction is traffic and load will make your application much more robust, responsive and resilient.
Why even bother with the API Key approach? It is useful for testing scenarios and for times when you just want to issue curl calls by hand, directly to the API. In these cases, it is much nicer to just do the call once, rather than having to get a token and then use the resulting token (which can be large).
How Do I Do It?
Set up a small section of code to request your API Token, and then save that token and insert it into every API call made to services on the IBM Cloud. You’ll then need to decide on how you want to handle renewing your token, since every token is only good for 60 minutes.
You can either set a timer for 50 or 55 minutes and wake up to renew your token before the previous token expires. The other option is to just handle authentication errors in your application by refreshing your token and trying to execute your request again with the new token.
Sometimes I post about things where I have done some work and provided some deep thought, and I want to share my knowledge. Sometimes I just “pass things along”, when I find something that is technically solid and useful. This post is more of a “pass it on” post.
Many of my customers are beginning to seriously think about doing “Cloud Native” development in 2019, based on the questions that I get asked. When I say “Cloud Native” development, I am referring to the development of projects that are built on the Cloud, and produce applications and (micro)services that live on the Cloud. This isn’t about moving server workloads to the cloud, it’s about new development efforts.
These customers are asking me about tooling, approaches and techniques for doing software development on the Cloud. I would often point them to some of my blog posts on how to organize your IBM Cloud environment, or to something good on microservices. One other good site, which I thought was more popular, is the 12 Factor App guidelines.
The 12 Factor App site has a nice simple write up which reminds you a bit of the Agile Manifesto. It has 12 factors which you can see, and each has a short and quick description of what is important about that factor. it’s definitely worth a look – it takes a grand total of about 15 minutes to read through the whole thing. Some of the factors are more development focused (like Config and Codebase), some are operations focused (Dev/Prod parity and Build/Release/Run), and others are more architectural (Dependencies and Concurrency) in nature. The important thing to remember is that ALL project participants should be aware of, and know the importance of, each one of the factors.
So if you are one of those organizations that are looking at making a serious impact with Cloud Native development this year, I strongly urge you to take the time to read the 12 Factor App guidelines and let me know what you think. Then feel free to ask me how to accomplish any of these things in the IBM Cloud.
I blog when I have to answer questions that I want to more widely share the answers to. It’s also a good way to remember things before a turkey induced amnesia sets in (it’s a week before U.S. Thanksgiving).
Recently I have seen some questions on being able to get access to the data in an IBM Cloud GitHub project. I had just completed doing a quick internal activity for pulling information out of a GitHub Enterprise repository, so I figured that this would be simple. It was…. and it wasn’t. The IBM Cloud GitHub instance isn’t a GitHub Enterprise deployment, it is a GitLab deployment. The GitLab API is a little different from the GitHub Enterprise API. I managed to find a suitable Python package for using the GitLab API, and if you look at the resulting code, it is pretty simple.
So I have created a simple GitHub project (called IBMCloud_GitLab_CSV) that does a quick CSV export of issues from an IBM Cloud GitHub project. It’s a simple example, written in Python, that you should be able to use and tailor, to fit your specific needs. I use small Python programs like this to pull the open issues from a variety of projects, and then I can share the resulting CSV files with project managers and PowerPoint producers who want to report on these sorts of things.
If you need this type of capability, make sure to read the README file for this project, which instructs you on how to modify the code to point at YOUR IBM Cloud GitHub project, and tells you how to get an access token for yourself (which the program needs, in order to be able to log into your GitHub project).
If you want to improve on this example, or even create some type of generic tool for doing this type of thing, please join the IBMCloud_GitLab_CSV GitHub project and begin contributing to it.
So now if you would like to get a copy of my Generic Calendar project to play with for yourself, it is really easy. Just make sure that you have a Bluemix account, and that you have a linked DevOps Services account. Then just navigate to my project in DevOps Services (it’s the dtoczala|ULLCloudCalendar project). Once there, you can look at the README.md file displayed there, and look for the “Deploy to Bluemix” button. It looks like this:
Just press that button and you will get a project created in the DevOps services that is a fork of my original project. The automatic creation of the project will throw an error during the deployment, but you will be able to easily fix this. The error is due to a problem in the manifest.yml file, we are currently unable to create and bind services for our application through the manifest (see Sean’s question on this in the forum). You can easily fix this by doing three things:
In your DevOps services console, configure your Deploy stage – In your newly created project, press the build and deploy button, and then configure the deploy stage. You will add a job to the deploy configuration, a deploy job, that will do a “cf push” of your application. Then try executing it. It will still fail (because our MongoDB service is not present), but it will create a new Bluemix application for you. This is your version of the ULL Cloud Calendar app.
In your Bluemix console, add and bind the MongoDB service – This is straightforward. Just add the MongoDB service and make sure to bind it to your new application. When you add the service, Bluemix will ask if you would like to restage your application. Answer yes, and wait for the application to be deployed again.
In your Bluemix console, click on the link for your new app – just click on the link for the route to your new application.
Now once that little issue with the manifest.yml is cleared up, you will be able to share your Bluemix applications with the press of a button. Bringing up applications and capabilities in the cloud is getting easier and easier!
Working with my team is often fun and rewarding. I learn a lot from the people I work with, and I get the chance to try and learn new technologies all of the time, in an effort to solve real business problems. One of our most recent challenges is having the ability to have a “team calendar” that we can all update. We wanted to have some light weight way to coordinate our activities, and to keep on top of vacations and travel plans.
We wanted something that would work within the External Content widget of RTC, because we wanted to expose this calendar on our RTC dashboard. The dashboard is where we track our work, watch our progress, and document our progress on our key measures, so it seemed to be a logical place for the calendar to live. RTC doesn’t have any kind of native calendar ability, and it is something we miss. I considered just using a Google Calendar, but we dismissed that because often our calendar entries will contain sensitive information. So we wanted something that could be done within the IBM firewall. That led me down the path of creating a simple calendar application using Bluemix. IBM has a small internal implementation of Bluemix behind our firewall, so our simple privacy and security needs could be met by this.
I decided on a simple implementation using Node,js (which recently announced it’s own Node.js foundation). I thought about using Cloudant for the underlying datastore, but in the end I decided on using Mongo DB, because I didn’t want this to be an “IBM solution”.
Keep in mind that this is a lightweight solution, it uses the Sandbox plan for the MongoDB service, and the code is not expected to be robust enough for 50 or 100 people to use. It’s meant as a nice sample project, and one that could be useful to a small team. It’s not going to replace your enterprise calendar solution. It also uses the dhtmlxScheduler component, which has it’s own licensing concerns. dhtmlxScheduler Standard Edition is available under GNU GPLv2, so be aware of the implications of this.
Would you like to see how to deploy it for yourself? Then read on……
Deploying the Generic Calendar on Bluemix
You’ll need a Bluemix account with all of the usual capabilities. You’ll first want to grab the code for this from my Github project called ULLCloudCalendar, and save it on your laptop/workstation somewhere. Just hit the button to download the zip of the contents of the project. I developed this on a Linux box, so hopefully the character sets and encoding don’t screw you up too much. Once you have a copy of the code, you’ll want to login to Bluemix. Once there, you will create a new application using the SDK for Node.js runtime. Give the app a good name (like “AcmeCalendar“), and wait for Bluemix to create your skeleton app for you.
Once Bluemix is done, you should see you new application on the Bluemix console. Now you will want to go and click on the box to “Add a Service or API”. Scroll through the list of services until you come to the “MongoLab” service. Click on the icon for this service, and on the next screen, create a new instance of the MongoLab service (which is a cloud hosted Mongo DB). Make sure to give it a name that corresponds to the name of your application (like “MongoLab-AcmeCalendar“). Also make sure that it is being set up in the correct space, and that it will be bound to the correct application (in my case, the “AcmeCalendar” application). When you have checked everything, press the “Create” button to create your service.
At this point, you will be ready for that code that you copied earlier. Have the code in a directory by itself. Make sure that you have downloaded the Cloud Foundry CF Command Line interface, and have installed it on your computer. Open up a command line interface, and navigate to the new directory where your code lives, at the top level directory. This directory contains the manifest.yml, app.js, and package.json files. Now we’ll log into our Cloud Foundry instance, set into the right environment, and push all of this code up to the cloud.
Login to the Cloud Foundry instance.
cf api https://api.ng.bluemix.net
Login to your account space on the Cloud Foundry/Bluemix instance. Use your Bluemix ID for the owner (-o) and user (-u) parameters, and the space name of your space on Bluemix for the space (-s) parameter. After you do this, you will be prompted for your password.
Now you will want to modify the manifest.yml file to make sure that you can find your new project. Edit the manifest.yml file and change all of the “ULLCloudCalendar” entries to the name of your application (Acme Calendar in this example).
Now modify the package.json file to reflect the new name of your application as well. Edit this file and change the line with the name to your new application name.
Finally, you can now push all of your code up to the Bluemix infrastructure. Use the name of your application (which is Acme Calendar in my example)
cf push AcmeCalendar
Keep in mind that there are other ways to do this (using an Eclipse plugin is one of them), so do a little research and find out the method that works best for you. Once you do the cf push of your code, you will see Bluemix/Cloud Foundry do it’s work. At some point it will tell you that your application has been deployed in the cloud, and that it is running.
Accessing Your Calendar
On the Bluemix console for your application, you will see a link to your new calendar application. It will be something like https://AcmeCalendar.mybluemix.net (depending on your application name and the route that you have chosen). You can change the route, but that is a technique for a future blog post (it’s not that hard, I just don’t want to get into it here). Clicking on that link will launch you to the website where you can access your new calendar. Play around with it. Double clicking on a day will open a dialog box for adding a new event. Double clicking on an event will allow you to change or delete the event. It’s pretty simple.
Adding this to your RTC dashboard is pretty simple too. Just create a new tab on your RTC dashboard. Click on the caret next to the tab name and be sure to set the tab up to display widgets in a single column (otherwise the calendar becomes too small to be useful). Now click on the “Add Widget” button on the upper right of the dashboard. Select the “External Content” widget. Once the widget is displayed on the dashboard, click on the small pencil in the widget menu bar. You will now enter in
The height of the widget (try 550 pixels for starters, adjust it as you need to)
The refresh rate (go with a simple 5 minutes, or 300 seconds)
Once you do this, you should now see you calendar application right on your RTC dashboard. You can even navigate through the calendar and add/change/delete events.
How Does it Work?
If you’ve read this far, you have enough knowledge to be able to deploy a simple cloud based calendar for your team. If you want to get into the code, and possibly change and enhance this calendar app, then read this section.
The calendar has three big pieces that control everything. The first piece is the dhtmlxscheduler piece. The code for this component (which controls the calendar look and feel, and drives functionality) is in the public folder. I didn’t touch this stuff, but if you want to try messing around with the CSS files to change the look and feel of things, be my guest.
The next big piece is the code that controls the rendering of the HTML page. This is in the index.html file in the public folder. There are two important pieces of code in this file. There is the script. The script first will go and make some configuration settings to the calendar, it then sets up the basic colors used, and sets up the look of the dialog box to add/modify/delete events. The script then initializes the calendar component with these settings. At the end of the script, the default date/time format is specified, your existing data is loaded, and a data processor is set up to handle the interactive user requests. Then there is the body of the HTML page, which sets up the display of the calendar itself, and initializes things. I didn’t fool around with this section.
The final piece is the app itself, in the top level directory in the file app.js. This file handles the storage and retrieval of data from the MongoDB, and does some data checking and data manipulation to help format things appropriately.
The script starts out by setting a bunch of global variables and reading the various VCAP settings provided by the Bluemix environment. This allows the application to connect to the MongoDB that is bound to this application with the correct credentials, and it also provides some other important runtime information. You will notice that there are references to SSO and certs that are in the code, but have not been tested.
Once this initial code is complete, you can see where we connect to the MongoDB. Following this is a section of code that is NOT tested (and not used) that deals with the SSO and passport functionality. This all ends up with the section on customer authentication middleware.
Finally we get to the Application routes.
The code for the /init route is simple, it just adds a single event on New Year’s Day to get you started.
The get code for the /data route supplies the calendar object with all of it’s events from the MongoDB. It retrieves ALL of the events in the datastore, builds a JSON object with these events, and provides them as a stream of JSON data to the calendar object in a response. Be careful with the formatting of your dates in the JSON response, an invalid date can cause problems.
The post code for the /data route processes the creation/modification/deletion of events by the calendar object. A user who changes something in the calendar will post the change to the /data route. This section handles the request, and processes it accordingly.
Finally at the end of the app.js file, we start the app.
There are things you can do to change how this calendar works, and expand or change it’s functionality. I’ll cover a couple of the simple things here.
I Hate The Colors
You hate the colors that I picked? So change them yourself. There are two sections that deal with the colors in the calendar interface. In the index.html file, there is a variable called colorpicker. You can change the names of the colors by changing the label property of the array entries, or the color itself by changing the key property of the array entries. This key property defines the RGB mix of colors. I used the HTML color picker to get these values. You can even add even more colors by adding more entries to the array.
These key values (the “rgb(x,y,z)” entries) are stored with the events in the database. If you look at the code in the app.js file, look at the get /data section of code. In here you see a section of code where we check the color property of the returned event data. This represents the color of the event box. Based on this, the if statement will either assign a grey box and black text (if no color information is provided), or the proper color and black text, unless the box is indigo in color, in which case white text is selected because it shows up better. Kind of hard to explain – easier to see in the code.
color = 'rgb(204,204,204)'; // grey block
textcolor = 'rgb(0,0,0)'; // black text
color = data[i].color;
textcolor = 'rgb(0,0,0)'; // black text
if (color == 'rgb(77,77,184)') // if block is indigo
textcolor = 'rgb(255,255,255)'; // go with white text
This one is an easy change in the index.html file. Just find this line of code:
scheduler.config.start_on_monday = false;
and change it to:
scheduler.config.start_on_monday = true;
I wanted a calendar that was stand-alone, that could be displayed in an RTC widget, and that I could easily deploy in a Bluemix environment. I hope this guide has shown you how easy this is to do, and allows you to add this calendar ability to your RTC environment. If you have comments or issues, please comment and I will do my best to answer your questions.b