Happy Holidays for 2017

With the end of the year quickly approaching, it is a great time to look back on the past year, and to look forward in anticipation for what is coming in 2018.

2017 was an interesting year.  I saw an explosion in the development of chatbots of various different types.  Some were very simple, others used both Watson Conversation and the Watson Discovery service to provide a deeper user experience – with an ability to answer both short tail and long tail questions.  I saw a huge uptick in interest in basic Cloud approaches, and a lot of interest in containers and Kubernetes.  I expect that both of these trends will continue into 2018.

In 2018 I expect to see the IBM Cloud mature and expand in it’s ability to quickly provide development, test and production computing environments for our customers.  I also expect that more people will become interested in hybrid cloud approaches, and will want to understand some best practices for managing these complex environments.  I am also excited about some of the excellent cognitive projects that I have seen which could soon be in production for our customers.  I also expect that some of our more advanced customers will be looking at how cognitive technologies can be incorporated into their current DevOps processes, and how these processes can be expanded into the cloud.

I hope that your 2017 was a good one, and I hope that you have a happy and safe holiday season.

Getting Data from your IBM Cloud GitHub Project

Note: This post is also published on developerWorks, as Getting Data from your IBM Cloud GitHub Project.  Please refer to that article to catch any updates.

I blog when I have to answer questions that I want to more widely share the answers to.  It’s also a good way to remember things before a turkey induced amnesia sets in (it’s a week before U.S. Thanksgiving).

Recently I have seen some questions on being able to get access to the data in an IBM Cloud GitHub project.  I had just completed doing a quick internal activity for pulling information out of a GitHub Enterprise repository, so I figured that this would be simple.  It was…. and it wasn’t.  The IBM Cloud GitHub instance isn’t a GitHub Enterprise deployment, it is a GitLab deployment.  The GitLab API is a little different from the GitHub Enterprise API.  I managed to find a suitable Python package for using the GitLab API, and if you look at the resulting code, it is pretty simple.

So I have created a simple GitHub project (called IBMCloud_GitLab_CSV) that does a quick CSV export of issues from an IBM Cloud GitHub project.  It’s a simple example, written in Python, that you should be able to use and tailor, to fit your specific needs.  I use small Python programs like this to pull the open issues from a variety of projects, and then I can share the resulting CSV files with project managers and PowerPoint producers who want to report on these sorts of things.

If you need this type of capability, make sure to read the README file for this project, which instructs you on how to modify the code to point at YOUR IBM Cloud GitHub project, and tells you how to get an access token for yourself (which the program needs, in order to be able to log into your GitHub project).

If you want to improve on this example, or even create some type of generic tool for doing this type of thing, please join the IBMCloud_GitLab_CSV GitHub project and begin contributing to it.

Watson as a Service

As I have said on earlier blog posts, if I have to answer a question more than once, then it’s probably worthy of a blog post.  This week I had one of those situations, and thought that it would be good to share because it highlights a few points at very different levels.

Last week I was answering a survey posted by a Watson Speech-to-Text user, who was complaining about how confusing everything was.  I found this a bit surprising, as Speech-to-Text is one of the more simple services for developers to work with.  The user claimed that their EasyVSL app was telling them that “Watson wasn’t working”.  So I quickly checked the Bluemix status board, and found that nothing indicated a loss of service availability.  So I asked some more questions and probed a little deeper.

Watson as an Add-On

I found out that the EasyVSL app has a feature which the user can turn on, which will use the Watson Speech-to-Text service.  The user just need to add the service credentials for a valid Watson Speech-to-Text service, and the EasyVSL app will use that Watson service instance to do their speech-to-text processing.  Once I saw this, I walked through creating an instance of the Watson Speech-to-Text service for the customer, and showed him how to get his service credentials, which he then supplied to the EasyVSL app.  Once this was done, Watson was no longer “broken”.

Up until the past few weeks, I had never seen a product use Watson services in this way – as an add-on to an existing capability, where the CUSTOMER needs to provide the Watson service.  In this case, EasyVSL doesn’t worry about paying for Watson services.  If their customers see value in them then they use them and pay IBM directly for the services that they use.  It is a business/operating model that I am beginning to see now.

Advantages and Disadvantages

This type of model has some positives and negatives associated with it, and I think that it depends on your perspective.  As an application provider, it is a positive to be able to let your customers pay for the additional functionality that they want.  If they don’t need advanced cognitive capabilities, then why should they pay for them?  It also allows you to avoid thinking about how your are going to pass charges along to your customers.  If you need to monitor customer usage, track how many times they are using the various Watson API’s, and then figure out some way to pass along the costs associated with them, you will end up spending a fair amount of time and effort developing that monitoring and billing infrastructure (not even considering the adjustments that you will need to make to make your business model work).

From a customer perspective, many of the positives become negatives.  While I like to have a choice as a customer, I do not like having to deal with two vendors billing me for a single service.  I am also not thrilled with the fact that I need to maintain an instance (or multiple instances) of the Watson services.  I am paying money to a vendor so THEY can worry about that stuff.  The other downside is that as a customer, it is not always apparent which vendor I need to call when I have a problem.  That was the issue that I saw this week – the customer that I had helped had reached out for support from both EasyVSL and IBM – and had not gotten very far with either of them.

Future of Watson as an Add-On

I’m not sure if this pattern will become popular, although my gut tells me that it won’t.  Most customers don’t like paying multiple vendors for what they perceive as a single service.  However, I do note that cell phones are now largely unsubsidized in the US, with customers buying their own phones, and then paying monthly for services that those phones use.  In the past, the phones were “free”, and the cost of the phones was “bundled” into the monthly cell phone rate.  So there is precedence for this type of model working.

If you are an app provider, it is critical that you think carefully about your approach to how you use the Watson cognitive services, not only from a technical perspective, but also from a business perspective.  The business model that you choose will have an impact on your development, operations, and your customers.

Multi-Cloud Strategy for Your Future

Those of you who read my posts know that I work for IBM, and that I work in the Cloud and Cognitive spaces.  Recently I have seen some articles on Multi-Cloud Strategies (like this one from the O’Reilly site), and it puzzles me why so many of the things that I see on this topic seem to miss some of the most basic things.  The O’Reilly article simplifies things, and misses some very important points.  I don’t blame them, it’s hard to boil down a complex concept into a blog post that can be read in 5 minutes.  So after explaining this more than once to someone, I figured it was time to write about it.

Getting Started with Cloud Computing

Cloud computing should be thought of as a tool – a tool to squeeze more computing power from a limited budget.  At the highest level it allows a customer to stop buying and maintaining their own sever hardware, and “lease” this capability from a Cloud vendor.  Now there are multiple things that you need to account for: security, availability, service availability (what kind of services are available), reliability, cost, etc.  Due to some of the current limitations of cloud technologies, not ALL workloads are suitable to be moved to the cloud.  There is a lot to understand, and it can be overwhelming.

For this reason, many companies decide to move slowly to the cloud, so they can learn and gain experience as they go.  This is a great way to learn about what is important, learn about the limitations and concerns that you should have, and to get some experience with cloud.  So they select a single cloud provider, and try a couple of pilot projects, moving some portion of their production computing to the cloud.

This is what I would consider a sane and reasonable approach to starting your journey to the cloud.  This approach reduces your risk, and allows you to learn as you go, gaining valuable experience and expertise as your initial projects get deployed.  Your team will understand he capabilities of your initial cloud vendor, both positive and negative, and will begin to become more comfortable with the whole cloud paradigm.  At this point you are not able to leverage the strengths of various cloud vendors, but you don’t really care because you are in learning mode.

Branching Out

Once these initial pilot projects have been completed, you will then branch out to doing some other projects with your original cloud provider, as well as doing some pilot projects with a second cloud provider.  This is the next step on your journey to a true multi-cloud strategy.  This second cloud provider will seem quite strange, they may have some different terminology, different billing, and different capabilities.  At first your teams will not like them, because they are comfortable with what they already know from your initial cloud provider.  Give them time to learn – over time your teams will get a more balanced view of the positives and negatives of the cloud vendors.

At this point you should begin building your own knowledge base of the types of use cases and workloads that work particularly well for each cloud vendor, and you should have some kind of rough order of magnitude way to price workloads that have moved to each cloud vendor.  Your team should also have a much better understanding of cloud concepts, and understand the differences in terminology between the cloud vendors.

A True Multi-Cloud Strategy

Once you have branched out to a couple of cloud vendors, branching out and evaluating additional vendors should be easy.  Your team will be experienced and will understand some of the key capabilities to look for, the things that are important for your organization.  You will be able to intelligently pick and choose between cloud vendors, based on price, capability, reliability, and based on your own experience.  You’ll be able to make well informed decisions based on your use case, the workload, and choose the best cloud vendor for that particular job.  So let’s look at what the positives and negatives of this approach are.

Negatives

  • Time – who wants to wait for two years while your team gets the knowledge that they need?  Why not just pick a single cloud vendor and be done with it?
  • Administration – monitoring and managing one cloud supplier can be a pain – managing 4 or 5 of them can be 4-5 times more painful.

Positives

  • Reduced Risk – your investment portfolio should be well diversified, it reduces the risk you face should one of you investments fail.  Your portfolio of cloud technology providers should be diversified for the same reason.  It allows you to quickly take advantage of new capabilities and technologies, and gives you FLEXIBILITY.  If you remember nothing else from this post – remember this point.  It is what all of the other articles that I see miss – and it is the single most important benefit.
  • Avoiding Vendor Lock In – have a cloud provider that is charging you too much?  If you’re familiar with other vendors, moving workloads between cloud providers can be easy (especially if you use container technology).
  • Best of Breed – giving your business the flexibility to choose a cloud provider that best fits their needs allows you to avoid overpaying for capabilities that might go unused with some cloud providers.
  • Best Practices – each cloud vendor may be strong in a particular area, but your team can generalize these best practices across ALL of your cloud implementations.

So wrapping things up, I think that a multi-cloud strategy IS important.  I strongly believe that the benefits of reduced risk and flexibility far outweigh the costs in terms of time (and patience) and administrative overhead.  It also allows your IT teams to focus on your core business, and pushes many of the “bread and butter” IT tasks off onto the cloud provider.  This allows you to focus and innovate.

Bluemix and Watson – Getting Started Right

Note: This post is also published on developerWorks, as Bluemix and Watson – Getting Started Right.  Please refer to that article to catch any updates.

In my role as a Watson and Cloud Adoption Manager, I often talk to customers who are new to the IBM Cloud platform (Bluemix) and the Watson services.  Often I will spend a good hour or more talking to customers and answering questions about the best way to get started with the organization of the Bluemix environment, and some of the more operational concerns around cloud and cognitive development.  Since I find myself answering these questions over and over, I figured it would be good to get this down in a blog post so I could easily share this information with MORE customers.  It also helps me remember all the important things when I do talk on the subject.

Getting Started on Bluemix

Signing up for Bluemix is easy, all you need to do is supply a valid email ID which will be associated with the Bluemix account.  This is fine for YOU, but what about your ORGANIZATION?  When setting up a Bluemix account for your company or organization, it is best to use a functional ID associated with your company.  Just keep in mind that the IBM Cloud will be sending out automated emails to this account (warnings about service usage, services being deprecated, new services available, and other things).  So you will want to make sure that you choose an email address that is monitored by someone, so you don’t miss any important notifications.

I KNOW it takes a little longer to set up a functional account within your company, and that it is a pain to go through the paperwork to justify it, but not having your Bluemix account tied to an individual will save you A LOT of time, effort and frustration in the future.

Learning the Cloud Concepts

This is a good spot to pause for a second, and take time to understand some of the basic cloud concepts that will impact how you organize yourself on the IBM Cloud.  Looking at this picture may help:

A sample of Bluemix organizations under a single account.

The outermost grey box is my Bluemix Account, and this should be associated with a functional ID.  This is the functional ID and email address where you will receive communications from the IBM Cloud team.  It is also the account which has access to EVERYTHING underneath it.

Inside of that grey box are three blue boxes.  These are the Bluemix Organizations, which represent different functional organizations (or projects) within the larger Bluemix Account.  Even though the different organizations reside in the same parent Bluemix Account, the resources within those organizations can be limited to visibility to only resources within that Bluemix Organization.

The green boxes inside of each Bluemix Organization represent Bluemix Spaces.  A space can be thought of as an individual development environment, or development area, for the development of a cognitive application.

Finally, at the lowest level, are the various Bluemix Services, indicated with the orange circles.  These can be Watson based services (like the Watson Conversation service), infrastructure services (like the OpenWhisk service), or other Bluemix capabilities (like the Weather or Blockchain services).  Services essentially “live” within a particular space, and the charges associated with a service are calculated and billed against the Bluemix Account.  These charges are broken down by Bluemix Organization, for your own internal billing and tracking purposes.

Getting Organized on the IBM Cloud Platform

We’ll assume that you took my advice, and have a generic corporate ID (like IBMcloud@acme.com) that is your Bluemix account.  Now let’s get organized to support the development efforts of your organization on the IBM Cloud platform.  I’m also going to assume that you have read the great guide to getting organized on the IBM Cloud Platform.  It’s good, and it has more detail than what I cover here.  It also doesn’t tell you how YOU should set up YOUR IBM Cloud Platform.  I’ll give you a “default” setup that you should use as the basis for your strategy in setting up the IBM Cloud Platform, and I’ll indicate options you have and why you might use those options.

We’ll start with our Bluemix account for our fictional Acme Corporation (for this example, we’re using IBMcloud@acme.com).  The first thing that we need to do is to create some Bluemix Organizations for ourselves.  Now my fictional company is like a lot of other companies out there, it has some divisions, and these divisions don’t often share responsibilities or development assets.  So I will start by giving each of the divisions that are looking at doing cognitive development their own Bluemix Organization.

I do this from the command line, using the Bluemix command line interface (CLI).  I go over some of the uses for the Bluemix CLI in Getting Bluemix Information for Support and Automation.  We want to create some new Organizations within our account, and we also want to create some new Resource Groups.  Some of our projects may be hybrid projects, and may use services support by both models – so we need to have associated areas.  In this case, we’ll create spaces for the Tau project, the Epsilon project, and the Phi project.

Here is what our command line looks like:

# Set the Bluemix API endpoint
bluemix api https://api.ng.bluemix.net
#
# Login to Bluemix, with -u username -p password
bluemix login -u IBMCloudAcct@acme.com -p "mypassword"
#
# Create new Bluemix Organizations for Delivery, Sales, and Marketing
bluemix iam org-create Acme_Delivery
bluemix iam org-create Acme_Sales 
bluemix iam org-create Acme_Marketing 
#
# Print a list of all of the Bluemix Organizations for this Bluemix Account
bluemix iam orgs

The first command sets the API endpoint.  The second command gets you logged in.  The next three commands create your new organizations, and the last command lists all the Bluemix Organizations for your account.  Check out the documentation for Bluemix commands for more details.  You will now see a listing of all of your new Bluemix Organizations, along with your default organization.  Be sure to provide a unique name for your organizations, and PLEASE establish some kind of naming convention.  I used <company_name>_<company_division>, you might want to do this too.

What our Bluemix account looks like after creating our new Bluemix Organizations.

Why set up these separate organizations on the IBM Cloud Platform?  For a few reasons:

  1. It helps keep the work organized and segregated.  With these in place, and once we get spaces set up, we can limit user access to only certain areas.
  2. It allows us to see usage and billing details by organization.  I can see exactly what each organization is using, and how much of my total bill each organization is responsible for.

Also keep in mind that these Bluemix Organizations are just a way to organize your IBM Cloud infrastructure.  After reading the guide, you may decide to go with Bluemix Organizations that break up areas based on the roles of your users, or specific projects.

Getting a Subscription

To this point in our setup of the IBM Cloud for the fictional Acme Corporation, we have assumed that you have used a “free”, or “trial”, tier of services.  These give you enough resources to figure out what you would like to do on the IBM Cloud, and allowed you to build small demo applications based on the IBM Watson services.

Now since we are talking about doing “real” application development work, we’re going to need more capability than these trial and free versions of the services provide.  That means that you will need to spend some amount of money to host these various services and capabilities.  Check out this nice overview of the various IBM Cloud Account Types to get an idea of your options.  The capabilities and limitations of these options differ, so it is important that you understand them.

For our fictional Acme Corporation, we have decided to go with the popular subscription option.  To begin the process of getting a subscription, you will want to follow the directions on how to obtain a subscription on the account types page.  Make sure that your subscription is associated with the right Bluemix account (the one with the functional ID, remember that?).

Getting Your Development Environments Right

Now that we have our organizations all set up, and our subscriptions associated with the correct divisions within the fictional Acme Corporation, it’s time to set things up for our development teams.  Now I understand that some readers are working for smaller, more nimble organizations.  You may be fine with doing things in an ad-hoc manner, and that may actually be the best working model for you.  What follows is a broad outline of what larger organizations, seeking a better separation of duties, control of environments, and implementation of a more standard software development lifecycle and DevOps culture will want.

We begin with our Bluemix Account and our Bluemix Organizations.  For our purposes here, we will focus on just one of our fictional Acme divisions, the Delivery division.  You begin by identifying the projects within the division, and you then begin to create Bluemix Spaces for each project.  You should also find out what environments each project will need.  Most will need a development environment, a test environment, and a production environment (at a minimum).  Some projects will want support for additional environments.

For each project, go out and create the appropriate Bluemix Spaces for the project.  Spaces need to have unique names.  For the sake of keeping things easily identifiable, unique, and organized, I have followed a naming convention of <Project>_<Environment> (you should probably do the same).  So for the Acme Delivery division, we have three projects – called Phoenix, WH, and Dizzi.  Each has different services that they use (some are common), and the Dizzi project has no need for a pre-production environment.

So now we use the Bluemix CLI to go and create the spaces that we need (Note: This can also be done interactively from the Bluemix UI – but I know that you are interested in automating this, so I will focus on using the CLI):

# Set the Bluemix API endpoint
bluemix api https://api.ng.bluemix.net
#
# Login to Bluemix, with -u username -p password -o organization
bluemix login -u IBMCloudAcct@acme.com -p "mypassword" -o Acme_Delivery
#
# Create new Bluemix Spaces within the Acme_Delivery organization
bluemix iam space-create Phoenix_Dev -o Acme_Delivery
bluemix iam space-create Phoenix_Test -o Acme_Delivery
bluemix iam space-create Phoenix_PreProd -o Acme_Delivery
bluemix iam space-create Phoenix_Prod -o Acme_Delivery
bluemix iam space-create WH_Dev -o Acme_Delivery
bluemix iam space-create WH_Test -o Acme_Delivery
#
# do the remaining spaces in a similar manner
.....

This results in a Bluemix environment within the Acme_Delivery organization which looks like the diagram below.

Details of the Acme_Delivery organization on Bluemix

You can double-check your results either from the Bluemix CLI (using the “bluemix iam spaces” command), or from the IBM Cloud dashboard.  You will notice that there are some small yellow circles within each newly created space.  These represent the IBM Cloud services that each project will create within these spaces, in order to implement their project.  As a Bluemix administrator, you should not have to do this.  The creation of needed services should be left to the individual project teams.

Getting Everyone Else Onboard

So now that you have your Bluemix Organizations, projects and environments (Bluemix Spaces) all set up, you are ready to go.  All that you have to do now is get your developers and other stakeholders into the environment.  The first step is to get your staff to register for Bluemix and to get their own accounts.  I STRONGLY suggest that your force them to register with their company email addresses.  It will make things easier for you to administer things as time goes on.

After your team has registered, you can invite them to the correct Bluemix Spaces and Bluemix Organizations.  So once again we will use the Bluemix CLIand send the invitations that we need (Note: This can also be done interactively from the Bluemix UI – but I know that you are interested in automating this, so I will focus on using the CLI):

# Set the Bluemix API endpoint
bluemix api https://api.ng.bluemix.net
#
# Login to Bluemix, with -u username -p password -o organization
bluemix login -u IBMCloudAcct@acme.com -p "mypassword" -o Acme_Delivery
#
# Send invitations to the various team members within the Delivery division
# arguments are: USER_NAME ORG_NAME ORG_ROLE SPACE_NAME SPACE_ROLE
bluemix iam account-user-invite john_smith@acme.com Acme_Delivery OrgAuditor Phoenix_Dev SpaceDeveloper
bluemix iam account-user-invite jane_doe@acme.com Acme_Delivery OrgAuditor Phoenix_Dev SpaceDeveloper
bluemix iam account-user-invite tom_tester@acme.com Acme_Delivery OrgAuditor Phoenix_Test SpaceDeveloper
#
# do the rest of your users in a similar manner
...

This gets your users added to the proper organizations and spaces.  Some important things to keep in mind when assigning organization roles and space roles to your users.

  • Give everyone OrgAuditor for their ORG_ROLE.  This allows them to see what is going on at the organization level, but doesn’t allow them to change anything.  The only exceptions to this are the people who do administration of the organization (your Bluemix Administrator), and handle the billing associated with the organization (often the Bluemix Administrator as well).
  • Space roles are a bit different.
    • SpaceManager: This role can invite and manage users, and enable features for a given space.  You may want to give this to the development lead for the Development spaces, the test lead for the Test spaces, and your operations leader for the Production spaces.
    • SpaceDeveloper: This role can create and manage apps and services, and see logs and reports.  Give this role to developers in the Development spaces, but you probably don’t want anyone with this role in the Testing and Production spaces.  This allows you to maintain the stability of these environments (no code changes).
    • SpaceAuditor: This role can view logs, reports, and settings for the space.  This is used for people who might be interested in the development efforts in a given space – but not requiring access to change anything within the space.

You will probably want to automate this, or provide a self-service capability, so people can easily request access to the environments that they need.

Herding the Cats

At this point we have our Bluemix environments set up, and we have our users added into the Bluemix Organizations and Bluemix Spaces where they need to be, with the access that they require.  Our fictional Acme environment looks a little something like this:

Overview of Bluemix Orgs and Spaces for our fictional Acme company.

Each of our divisions now has the ability to control access to their Bluemix areas, and can effectively isolate their various environments.  Each is free to use whatever promotion process and DevOps tooling they prefer, or they can utilize the IBM Cloud Toolchains and the IBM Cloud Continuous Delivery service.  Use a DevOps solution which addresses ALL parts of your development process, regardless of where they reside.

There are some important things to consider when thinking about the topic of DevOps in conjunction with the IBM Cloud and the IBM Watson services.

  • Watson services where you do not provide any training data (Document Conversion, Language Translator, Personality Insights, Tone Analyzer, Text-to-Speech (TTS) and Speech-to-Text (STT) ), can be deployed to additional Bluemix Spaces and do not need to have their data migrated from one environment to the next.  Keep in mind that on occasion the versions of the API and underlying services may increment, so you will want to control which API endpoints are used for these services.  (Note: If you have customized either Text-to-Speech or Speech-to-Text, then these services will need to be migrated from environment to environment.)
  • For Watson services with training data (Conversation, Discovery, Natural Language Classifier (NLC), Retrieve & Rank (R&R), Natural Language Understanding (NLU), and Visual Recognition), you will need to deploy these to environments and migrate their data.  There is currently no mechanism in place to copy service instances from one space to another.  This has some very real implications beyond the operational aspects.
    • For services like Conversation which support an export/import functionality, it means that you can choose to do your training through the REST API, or interactively through the tooling provided on Bluemix.  When you are ready to move an instance from one environment to another, you export from one environment, and then create a new instance in the target environment, and import your data.
    • For services like Discovery, which do not have an export/import capability, you should resist the temptation to train “by hand”, and instead have a script (or series of scripts) that can be used to train your system.  In this way, you can more easily create/recreate an instance of the service.

Conclusion

This should give you enough of an overview, and offer you links to enough information, so that you can begin using the IBM Cloud platform to develop cognitive applications with confidence.  Technology changes rapidly, so if you see problems with this basic approach, or have some best practices to share, please reach out to me.

Good Reference Materials

  • Setting up your Bluemix environment – Not a lot of guidance here, but this documentation page on Bluemix has a lot of the basic information that you will need to understand when setting up a DevOps supported development environment for your cloud and cognitive development.
  • Bluemix CLI – The reference page for the Bluemix specific CLI.  Please use the Bluemix commands and not the Cloud Foundry equivalents, to ensure smooth operation of your Cloud environments.

Save

Full Cycle Cognitive Development – Part 2 – Agile Development

Introduction

In part 1 of this series, Full Cycle Cognitive Development – Part 1 – Business Concepts, I talked about some of the simple organizational issues and concepts that are important to know when doing cognitive development.  I am an ex-developer, so this second part is a bit closer to my heart.  In this blog we’ll look at some of the basic “blocking and tackling” that developers need to do as part of a successful cognitive development effort.

Use Agile development methods

Most software development teams use some sort of Agile methodology today.  It’s not perfect, and there are “shades” of Agile.  Some teams are able to fully embrace Agile development methods, but others cannot due to business or regulatory constraints, or safety concerns.  There are thousands of articles, papers and blog posts about Agile development – and everyone has sightly different approaches and opinions, so I won’t go into great depth here.

What I will say is that the development of cognitive solutions is often a very fluid process.  IBM (as well as other vendors) often come out with new cognitive capabilities which you will need to assess and evaluate.  Don’t try to lay out a full blown project plan without acknowledging that the changes in this area are going to be constant.  New capabilities may emerge that can change the way your solution is constructed, or may even change the original scope and goal of your project.

Because of this constant evolution and change in the capabilities of cognitive services, it is essential that you embrace the idea of sprint planning, and the basic philosophy of Agile development.  If you find yourself beginning to produce Gantt charts, detailed requirements documents, and calling out delivery dates 9 months in the future, you’re probably moving in the wrong direction.  Focus on user stories, sprints, and the delivery of incremental value.  Keep in mind that your business goals probably won’t change too much, but the technological path that you take to achieve these goals will definitely change over time.

It’s All About the Data (Science)

Another reason Agile is attractive for cognitive development efforts is because cognitive capabilities sometimes require training.  Some capabilities come pre-trained, where the vendor has already spent the time and effort to train and model the cognitive service.  Others will require you to train your service, teaching it how to react and respond.

These trained services will require data in order to be trained, and the accuracy of the trained service will need to be assessed over time.  This training is a bit non-deterministic, so it could take 2 sprints, or it could take 10 sprints, to get your service adequately trained.  James Ravenscroft has some excellent blog posts about training your cognitive service, and then testing it out, which I suggest you read.  If you have the time, read some of the blog posts on Cognitive System Testing by Andrew Freed – he’s really good and he knows the subject area quite well.  He has a great post on Reaching Peak Cognitive Performance that I consider a “must read”.  Another good source on this is Marc Nehme, who has some good guidance on training the IBM Watson services.  (Author’s note: I work for IBM on Watson, so most of my links/references will focus on the Watson technology.  The basic concepts here should hold for most AI vendor technologies.  If you know of other good sources, please reference them in the comments to this blog post.)

One common area of confusion is around the testing of cognitive services and trained systems.  This is NOT traditional application testing.  Traditional applications are deterministic, if you do “A”, they will respond with “B”.  Cognitive systems (and AI systems in general) work with probabilities – and they are imperfect just like you and me.  So testing requires a bit of a different mindset with cognitive systems.  With cognitive systems, 100% accuracy isn’t realistic.  If all of this sounds “just plain wrong” to you, then I strongly encourage you to read the testing links in the previous paragraph.  Those folks explain it much better than I could.

All of this training and testing rely on one thing – data.  It sounds simple, but there is a science to it – which is why we have data scientists and the discipline of data science.  A cognitive system is only as good as it’s training, and it’s training is only as good as the data being used to train the system.  My team has worked with hundreds of organizations looking to build cognitive applications, and my architects always seem to come back to a single point when assessing the prospects for some new effort – “How good is their data?”.  Data is critical.  Data is king.

So how do you get your hands on the data, and once you do, what should you do with it?  My advice is to get a good data scientist (or multiple data scientists) to help you.  They know how to get information and intelligence from both open source data sets, as well as your own private data sets.  You can also check out the IBM Data Science Experience, which has all kinds of resources that you can use.  The site has articles, some sample data sets and notebooks, tutorials and more.  It can be a great place for you to begin to understand how you can think about your data, and figure out how to use this knowledge to help drive a cognitive application.

The key to keep in mind here is that as your learn about the cognitive capabilities that you have at your disposal, and as you train these services, the feedback that you get can cause you to change your focus and schedule.  Data sources may change, training may take longer than expected, additional data may have to be obtained, and existing data may need to be annotated.  Stay disciplined in doing sprint demos and backlog grooming.  These activities will allow you to show positive progress and keep you aligned with your stakeholders.

Use tools

This is where a lot of Agile people get confused.  Agile teams don’t HATE tools, they just don’t use tools unless doing so adds value.  The Agile Manifesto states that, “We value individuals and interactions over processes and tools”.  It doesn’t say, “We hate all tools”.  When doing cognitive development, you will need to rely on tools to make your job easier (valuing the time and energy of the individual), and to show your stakeholders what you are doing (helping keep interactions based on reality).

So what kinds of tools are useful here?  Let’s cover a few of the different areas.

Agile Tools

There are hundreds of different Agile tools that you can use to help keep your cognitive development organized and focused.  You can use things like the IBM Jazz products, GitHub/ZenHub, VersionOne, Atlassian and others.  Some are open source, some are vendor provided.  Developer communities can get into long arguments about which tools are better.  Don’t waste a lot of time and effort doing tool evaluations, just choose the tools that your team will work the best with.  The tools that you select should provide a clear mechanism for your development team to communicate status, ask for help, indicate issues and problems, and allow you the ability to quickly find and focus on problems.  They should allow your team to do this without significant additional effort, and they should be something that is as unobtrusive as possible.  In other words, they should value the individuals, and not the process.

Whatever tools you choose should have some transparency and the ability to show team progress, goals, and challenges on a DASHBOARD.  If you’re a long time reader of my blog, you know that I find dashboards to be helpful not only in communication of status and issues, but they help teams become transparent, and allow you to spend less time arguing over who’s version of reality is correct.  The best Agile development teams that I have seen have been transparent, almost painfully so.

Cognitive and Data Tooling

Many of the different vendors of cognitive capabilities also provide some kind of tooling or scripts that help to make their services easier to use.  These range from simple tools that can be used to break up training and test data, to some more sophisticated data annotation tools (like Watson Knowledge Studio) and analytics capabilities.  Just like the Agile tools, some of these tools are open source and you can just grab the code from GitHub repositories, other things are vendor provided, and some cost money.

Data Tools

Often cognitive services will need data in a particular format, XML or JSON, before they can use the data.  Save yourself some time and trouble and just do a simple Google search for simple data translation tools like these.  More complex tools may require licensing or some cost.  You should also spend some time with your data scientist (you did get one, right?).

Make sure that your data scientists have the tools that they need to be effective.  It’s also a good idea to have your data scientists using the same tools – that way they can begin to reuse certain methods and techniques, and begin to build up a solid data science discipline in your organization.  You wouldn’t want your developers all working with different IDE’s, compilers, languages, on an ad-hoc basis, would you?  It would be total chaos.  The same thing holds true for your data scientists – be nice to them and they will reward you with insights and business intelligence that will amaze you.

Some of the more common tools that data scientists use include things like Python (along with the Pandas, SciPy and NumPy modules), Jupyter notebooks, R, D3 (D3.js), Spark, and TensorFlow.  Some of these are programming languages well suited to data manipulation and analysis, some are data visualization frameworks, and some are just data driven technologies.

Software Tools

For those of you who know me, this is an area near and dear to my heart.  We’ve already discussed Agile tools, but what about other software tools?  What else do you you need to be aware of?

You need development tools, but before you go and get some, keep in mind that most cognitive technologies depend on (or are deployed on) cloud infrastructure.  Keep in mind that the tools that you choose will need to work in a cloud environment.  If you’re working with Watson, then you will need to provision and access the Watson services on the IBM Bluemix cloud.  Do you want your application to live in this environment as well?  Or are you going to host your application on your own infrastructure, and make calls out to the Watson services?  Where is your data going to live?  Do you have private data being used by the cognitive application?  Are you using two types of data, with some data used for conversational and user context, and another set used for training your cognitive service?

With cloud environments, this can all get overwhelming pretty quickly.  So the first thing I would suggest is a good drawing tool.  Pick what you want, Visio, Mural, PowerPoint (don’t shoot me – some of you actually use it for this kind of thing), some architecture tool like Archi, or something else.  You just need to have the ability to easily express in pictures what your cognitive application is going to look like from a technology perspective.  A lot of people will want to know about specific pieces of your application (security, deployment, etc.), and you will want to communicate clearly with them.

Next you need to be able to actually write code and that means you’ll need some sort of IDE for your developers.  There is a lot out there to choose from, and you probably already have something that you are using.  If it’s not broke – then don’t fix it, and just let your developers use what they are comfortable with.  Some of the more popular IDE’s work for more than just one language, and that is probably a direction I would go in.  You can use something like Eclipse, Intellij, Xcode, Atom, Komodo, or even the Orion IDE that is part of Jazz Hub (more on that in a little bit).  Some of these are open source, some are vendor tools that you’ll need to pay for.  Find what gives your developers the most value, and go with it.  Just make sure that whatever you choose has some way to integrate information back to your dashboards (remember those from the section on Agile tools?).

You also need to consider Software Configuration Management (SCM) tools, so you can develop code without having developers stepping all over each other.  There are a LOT of different SCM tools in the market, and if your company is normal you probably already use multiple different SCM tools.  Developers can get “excited” about their favorite SCM tools – they love some and hate others.  You can select from things like the Jazz tools, Git or GitHub, Subversion, Razor, CVS and others.  Like all of the other tools I’ve mentioned, some of these are open source, some are vendor tools that you’ll need to pay for.  Recently I have seen a lot of work being done by my customers on either GitHub or GitHub Enterprise (think of it as a “private”GitHub).  If you’re using GitHub, then you might want to look at ZenHub for Agile tooling capabilities.  My own team has been using GitHub Enterprise with ZenHub Enterprise, which gives us SCM and Agile tool capabilities (but no real dashboarding).  I like it because it’s all handled out in the “cloud” somewhere, and it integrates with Jazz Hub for a cloud based, seamless, deployment to Bluemix.  We still need to use another technology for our dashboards, but it allows us to be flexible, responsive, and get up and going on any project pretty quickly.

I did mention Jazz Hub in my list of potential IDE’s.  That is a qualified recommendation, and it’s a bit different.  As far as an IDE is concerned, it’s a bit limited in it’s capabilities.  It does have some really cool features though, which is why I mention it.  It has some Agile tooling and dashboarding capabilities built into it, which you can access by hitting the “Track & Plan” button in your project.  You also have an integrated GitHub repository for your SCM needs.  Finally, it also has some built in DevOps tooling (more on that in my next post).  In my work with it I have found it sufficient for small projects, and it really simplifies things having everything Cloud based (IDE, SCM, Agile tools, deployment to Bluemix, etc.).  Give it a look (it’s free), and see if it works for some of your projects.

I can hear people already clamoring for sustainable software development, with DevOps tools and delivery pipelines.  Don’t worry, I haven’t forgotten about them – the topic is too big to approach here.  That is the subject of Part 3 of this series, Full Cycle Cognitive Development – Part 3 – Do It Again.

Get Started!

Now you should have most, if not all, of your supporting infrastructure in place.  Now you need to get to work actually creating something useful.  Less talk, more action!

Start by getting your epics and user stories collected.  They won’t be perfect, and they will change over time, but you need to get started.  Assemble your team, get them familiar with their working environment, and begin to do some sprint planning and prioritization of user stories.  Let your teams get started and begin to find a working rhythm, and have the whole team monitor the progress of their efforts on your dashboards.

You are going to make mistakes – be honest about it and accept that fact.  Your processes and tools will change as your team begins to add things to the process to ensure quality, and as they remove things from the process that add little or no value.  Your cognitive team is not only doing software development, they are doing cognitive system training, and that those two things are quite different.  You need to be flexible and understand that what works for other organizations might work for your organization, or it might not.  Don’t believe everything that you read (even this blog!), believe what you SEE and what you MEASURE in your own teams.

Conclusion

Cognitive development is new for most organizations.  It is NOT your typical software development effort, although it shares many of the same concepts and metrics.  Keep in mind that cognitive systems are trained, not programmed.  If you’re doing cognitive development, people are going to be watching you – it’s just the way that things are.  So spare yourself and your team the agony of endless status meetings by being transparent, and show your progress and struggles on a dashboard.  Embrace Agile development concepts, and don’t be afraid to replay, reprioritize, and adjust as time (and sprints) moves on.  Being transparent will give your management team some comfort with all of the change that cognitive development will bring, and will allow you to quickly introduce new methodology, metrics and concepts.

In part 3 of this series I will explore how you make cognitive development repeatable and part of your development culture, and the important role of DevOps concepts in making this a reality.

Getting Bluemix info for Support and Automation

Note: I added a section to this on April 10, 2017 on creating an org within your Bluemix account.  This was also published on IBM DeveloperWorks on April 13, 2017.

I like writing blogs, and I find that the inspiration for many of my blog posts is the fact that I hate answering the same question twice.  So when I was recently asked to help out a customer who needed some help in dealing with Bluemix support, I found myself answering a similar question to one I have found myself answering often in these circumstances.

The issue is that when working with Bluemix, the interface hides a lot of the complexity of managing a Cloud platform from the end user.  That’s good – I don’t really care what my org ID is (assuming that I even know exactly what an org ID is).  However, our Bluemix support teams often have to work “behind the scenes” in Bluemix, and they need to know things like org ID’s, instance ID’s, and service ID’s.  They will often ask for this information, and our customers have no idea how to get it.  So just follow along here, and you’ll be able to get all of that information (and more).

Setting Up

In order to get this information, you will need to use command line interpreter (CLI) tools.  There are two of them: the CloudFoundry CLI, and the Bluemix CLI.  In this example below, I show how to use the Bluemix CLI to get this information, but if you watch your terminal session, you can see how you would do things using the Cloud Foundry CLI, because the Bluemix CLI will tell you what Cloud Foundry commands it is invoking.

So to get started, you will need to install the CloudFoundry and Bluemix command line tools (download them here). 

Need to Setup a New Bluemix Org?

Some of you may want or need to set up a new Bluemix org under your account.  This cannot be done if you have a trial account, but if you have a pay-as-you-go account or Bluemix/Watson subscription, then you will be able to do this.  Just type in the following commands using the CloudFoundry command line tool:

  • Set the API endpoint
bluemix api https://api.ng.bluemix.net
  • Login to CloudFoundry
bluemix login -u <YOUR_USERID> -p <YOUR_PASSWORD> -s <YOUR_BLUEMIX_SPACE>

(I used cf login -u dtoczala@us.ibm.com -p myBig$ecret -s dev)

  • Now create your new org in your Bluemix workspace
bluemix iam org-create <NEW_ORG_NAME>

And then check to see if your new org was properly created.

Finding Out Your “Secret” Identifiers

Now you’re ready to find all of those obscure and hidden ID’s that support people are asking for.  Make sure that you have the Bluemix and CloudFoundry CLI’s downloaded and installed on your system.  Then you can do the following from the command line:

  • Set the API endpoint
bluemix api https://api.ng.bluemix.net
  • Login to Bluemix
bluemix login -u <YOUR_USERID> -p <YOUR_PASSWORD> -s <YOUR_BLUEMIX_SPACE>

(I used bluemix login -u dtoczala@us.ibm.com -p myBig$ecret -s dev)

  • Now see what orgs are out there in your Bluemix workspace
bluemix iam orgs

Check the list and get the name of the org that you are interested in

  • Now get more information on that org – just for fun
bluemix iam org <YOUR_ORG_NAME>

(I used bluemix iam org dtoczala@us.ibm.com)

  • Now get the Org ID information on that org – just add the –guid switch
bluemix iam org <YOUR_ORG_NAME> --guid

(I used bluemix iam org dtoczala@us.ibm.com –guid)

Copy/paste the returned information – this is your org ID.

  • Now lets get a service instance ID.  Start by getting a list of your services
bluemix service list

Get the name of the service in question (I pulled a Watson Conversation service)

  • Now get some more information on that service – just for fun
bluemix service show <YOUR_SERVICE_NAME>

(I used bluemix service show ToxSampleConversation)

  • Now get the Instance ID information on that service – just add the –guid switch
bluemix service show <YOUR_SERVICE_NAME> --guid

(I used bluemix service show ToxSampleConversation –guid)

Copy/paste the returned information – this is your service instance ID.

Summary

  It is pretty simple to get information about your Bluemix account using these CLI tools.  You can use them from a terminal, but they also work from within shell scripts, allowing you to automate reports and anything else that you might frequently want to do on Bluemix.  Check out the Bluemix CLI Reference page and the CloudFoundry CLI Reference page for more CLI commands that you can use to get information for your Bluemix account, and your Bluemix services.

60 Minutes talks Artificial Intelligence

There is a television news show called “60 Minutes” that runs here in the US.  It runs after our Sunday football games.  I usually don’t watch the show, and rarely comment on media, but this week they had a segment on artificial intelligence.  The first part of the segment talked a lot about the history of the Watson technology, including the use of Watson on Jeopardy, and some of the different tasks that it has been put to.  There was a big emphasis on the work done around Watson for Oncology.    What they talked about a bit, but didn’t really show, was the many different ways that Watson and other artificial intelligence implementations are working in the world right now.  The first segment was pretty accurate, with only one big glaring error.  They keep referring to Watson as a “thing”, almost a single sentient being.  Watson is actually a collection of services that express various capabilities of artificial intelligence.  (Authors note: It was also really cool to see some of my Austin co-workers, like Rob High, on network television.)

The second part talks about the future of AI.  They spend a lot of time looking forward at what AI can become, and they begin to have a discussion about the ethics of artificial intelligence.  They mentioned the recent effort of the five industry leaders (Microsoft, Amazon, IBM, Facebook and Alphabet) in the ethics of artificial intelligence.  They didn’t get into anything resembling the deep conversations being held right now about displacement of workers, AI ethics, and the ultimate control of AI.  There are a lot of issues here, and they touched on some of the more “scary” scenarios, without going into any depth.

I will be interested to see what kind of feedback and reaction this gets from the “general public”.  The early reactions range from, “Do these guys understand the dangers?  Don’t they read Azimov or watch movies?” (yes – we’re extremely familiar with these things, we’re not one dimensional geeks), to people who know something about the subject wondering why they didn’t discuss neural nets and deep learning (because it’s a 30 minute show meant for public consumption).  It all comes down to educating the general public on exactly what AI does, what is doesn’t do, and what it is capable of.  That will be a long mission.  Computers and software have been around for roughly 70 years, and it’s only in the past 10 to 20 years that the general public has come to a general understanding of those.  We have a long way to go……

What is a Watson Bootcamp?

Note: This blog has been reprinted on the IBM developerWorks blog, under the title “What is a Watson Bootcamp?“.  Look there for the latest edits and changes to the article.

Where Do We Begin?

Many people and organizations who look to adopt and use the Watson technologies will often struggle at first.  They struggle with the idea of getting started using IBM Watson and the cognitive technologies.  Some of it is the common fear of the unknown, some of it is just not knowing where to start.  Are there things I should read?  What order should I read them in?  When we talk about enabling an entire development team or organization, the questions get tougher.  Should everyone concentrate on the same things?

One of the key methods that I have seen my team use in getting IBM Watson customers familiar with the Watson capabilities, and in how to apply them to their business problems and challenges, is through the use of a Watson Bootcamp.  These used to be called Watson Hackathons – but “hackathon” can have a negative connotation to some people, so we now refer to these as “bootcamps”.

How Does a Watson Bootcamp Work?

A Watson Bootcamp is a two day, everyone at one site, hands-on session.  It is run by two Watson developers or architects, who lead the group.  We’ll call them your “Watson Coaches”.  The attendees are a collection of roughly 20 people (the size can vary), with a mix of developers, architects, data scientists and subject matter experts (SME’s).  This group is broken into teams of 4-5 people each – with each team having at least 2 developers and 1 data scientist.  Since these teams work collaboratively, they need to be together.  Remote attendance does not work for the bootcamp, everyone needs to be physically present at one location.

Bootcamp Preparation

We begin two weeks prior to the bootcamp, the Watson Coaches will discuss potential bootcamp use cases.  Typically we like to see a list of 6-10 different potential use cases to cover in the bootcamp.  The coaches will work with you to identify the 2-5 best use cases, and the ones that will be addressed during the bootcamp.  We will also agree on logistics for the bootcamp.  You will need to have the following things:

  • A location with good internet access, that can support the development work of all of your attendees.
  • A Bluemix account where the Watson services will be created.  Individual services will need to be created for each bootcamp team (so they don’t impact each other).
  • A large room at your location for kick off activities, and for the final read-out from each bootcamp team.
  • Smaller rooms for EACH team – a space where they can work together and collaborate.

Logistics Note: You will need to reserve one BIG room that will hold all attendees for the kickoff on day 1, and for the final presentations on day 2.  You will also need to reserve SMALLER rooms (one per hack team) that will be the home for each team during the hack phase of the bootcamp.

Preparations continue during the week prior to the bootcamp, with a general session for all bootcamp attendees.  In this initial session, usually done remotely via conference call or e-meeting, the Watson Coaches will do a quick overview of the various Watson services that may be applicable to the bootcamp, and will give the attendees links to some pre-Bootcamp educational materials (here is a sample of those pre-Bootcamp educational materials).

Yes – we actually expect attendees to go through these materials!!  If they do, it will pay off for them in the long run. (hint!)

Bootcamp Execution

Then we come to the first day of our onsite bootcamp.  We get all of the attendees into the same room, and we typically provide from 2-4 hours of additional training and classroom exercises.  We will also go over successful working models with the teams, and give them ideas for how to get started.

Once the morning classroom session is complete, the attendees break up into their teams, and each team will get their own use case to implement.  You may want to assign use cases to specific teams, or you might want to let them choose use cases on their own.  It’s up to you, and what is most effective in your corporate culture.  Each team will then go off on their own, and begin to build a cognitive application using the Watson services that will implement their use case.

Teams will spend the remainder of day 1 working on their use case, and will often work into the evening.  They come back on day 2 and continue to work on implementing their use case.  While the teams are separated and working on their use cases, the Watson coaches will circulate between the teams offering guidance, technical support, and some problem solving ideas to the teams.  Often teams will segregate the work based on their roles, with each team member doing what they do best.

Bootcamp Close Out

After lunch on the second day, each team will usually spend the last hour or two of their time preparing a short slide deck with a pitch based on their cognitive application, and fixing bugs.  The last 60-90 minutes of the bootcamp consists of a series of 10-15 minute presentations done by each team.  The presentation is essentially a pitch – showing a demo of what they have built, highlighting how it could be extended and improved, and explaining what they have learned.

The presentations are usually given to a panel of judges.  The judges are a mix of business and technical leaders from the customer, and they score presentations based on some criteria.  The usual criteria that we use include the following:

  • Ability to Extend/Expand – Does the prototype appear that it will be easy to extend or 
    expand, to provide additional value to the business?  
  • Shared Knowledge – Did the presentation help educate the wider boot camp audience? Did the team share lessons learned and personal observations on the use of the Watson cognitive services?  
  • Innovation – Is the prototype or the approach used to solve the problem new and innovative?  Is this something new that has value?
  • Feasibility – Is this something that could move from prototype to product in the next 6 months?
  • “Wow” Factor – Does the prototype grab the evaluator, and provide a significant business or consumer capability?
  • Demo – was the prototype smoothly working?  Was the scenario “canned”, or did the demonstration show real cognitive capability? 
  • Entertainment – was the presentation engaging and entertaining?  Will people remember this presentation a day from now?  A month from now?  Will it “stick”?

The judges will typically score each presentation, and these scores will be collected.  Based on the scores, a “winner” for the bootcamp will be announced, and all members of the winning team will receive some sort of prize.  The prize depends on your culture.  It could be an old repurposed bowling trophy, it could be cash, vacation days, or something else.  The key is to make it valuable enough to encourage healthy competition, but not so valuable that it encourages unhealthy competition.

Why Does This Work?

This bootcamp approach seems to work for a number of reasons.  I’ll list the major reasons that I have seen, but I am sure that someone having gone through this experience might have some other reasons to add.

  • People learn best by doing things – People can learn in a passive (lecture style) environment, but they seem to learn better, and retain information better, when they learn in a “hands-on” environment, where they learn by actually doing something.
  • Competition narrows focus – the friendly competition between the teams helps to focus them on learning as much as needed, and on learning what is applicable.  Teams are under time pressure, so they don’t waste time learning about obscure product capabilities, but instead they focus on the capabilities that have value to them.
  • Team collaboration allows teams to be efficient – With a mix of roles on each team, teams can have strong resources that can cover each part of cognitive development with Watson.  Developers can learn about the importance of data in cognitive systems from data scientists – without having to struggle with doing the data curation themselves.  SME’s can learn about cognitive development, but can focus their efforts on providing their expertise to the overall system.  It allows people to learn about everything – while working within the areas where they are skilled.

This is AWESOME!!  How Do I get One of These?

You could potentially run a session like this yourself, if you have the knowledgable Watson Coaches who can help guide your teams.  If you are looking to have one of these sessions using IBM’s Watson Coaches, just reach out to me (or have your IBM representative reach out to me), and we can see about getting a session set up.

I find that the people and organizations that execute these bootcamps tend to come away from them with a MUCH better understanding of Watson and it’s capabilities.  It also gets people thinking.  Usually a couple of weeks after a bootcamp, I will begin to see question related to their new ideas about how they can apply the capabilities of Watson to solving their business problems.  The bootcamp combination of education, competition, and “learning by doing” seems to get people thinking in new and innovative ways.

Learning About Watson – Getting Started

Note: This was originally published in August 2016, I have updated it since then, with the last update made in March 2017.

Recently I have done a few hackathons/bootcamps with some different customers.  A bootcamp involves a group of developers that are split into teams of roughly 5 developers per team.  Each team gets a use case, or real world problem, to solve.  We usually spend the first half day teaching the participants about cognitive computing and how Watson services can be used.  We then spend the next day and a half doing the “hack” part of the bootcamp.  At the end of the bootcamp, each team will present their prototype, and typically a winner is chosen.

In general, a bootcamp is focused on helping people understand Watson, and then having them quickly build a prototype application that uses Watson’s cognitive capabilities to solve some kind of real-world problem.  These can be hectic, with me helping and coaching teams looking to use Watson for the first time, but they are a lot of fun.  It’s really interesting to see teams of developers come up with new and innovative ways to solve problems, and I always find it exciting to watch the teams focus on solving real world problems.  Since people learn best by DOING, it also serves as a great way to teach people about Watson, how to use it, and what it can do.  Read my post on What is a Watson Bootcamp?, to learn more about these.

After this last bootcamp I had a discussion with one of my customers on how to “best learn about Watson”.  We discussed how bootcamps can be really effective, since they force people to overcome that early apprehension whenever trying something new, and because people are able to get their “hands dirty” solving real problems.  He then asked me, “You can’t expect to run 1000 bootcamps this year – how do the people who can’t do a bootcamp learn about Watson?”.  It was a good question, and I realized that my answer was something that a lot of people might be interested in, so I figured I would share it here.

One of the first places that is good to start is relatively new, it is the IBM Learning Lab.  The site has a bunch of great resources for learning about Watson on your own.  If you’re one of those people that likes online courses, you can look at a catalog of cognitive courses that are available to you.  It’s a mix of courses from Coursera, Big Data University, Codecademy and the IBM Watson Academy.  You can browse courses based on your skill level.  Some of the courses are free, others require you to pay for them.  You can learn about Machine Learning, Data Science, Watson technology and more.  It’s a nice way to pick and choose learning opportunities for yourself.

Personally, I am more of a “hands on” learning type of person.  One of the other great features of the IBM Learning Lab is the section on Use Cases.  This section has a bunch of use cases focused on a particular Watson capability, a particular industry sector, or on some type of function.  The Pokemon Go use case is one of the most interesting (and maybe least applicable to a business problem), but there are others focused on things like virtual agents (or bots), social media analysis and response, automated systems, and others.  Just knowing the use cases is helpful, but what really makes this great is the fact that the code for an application to address each of these use cases is also available.  Having this code available means that you can look at the code to see HOW to use the Watson capabilities, and you can even use these as a rough starting point for your own cognitive application.

One other great place for getting cognitive use case ideas, and the code that implements an application to solve the use case, is the Application Starter Kits (ASK’s) page out on the Watson website.  You can look at the use cases presented here, and find a use case that helps address a real problem that you have.  These also have code available for you to use.  Often in hackathons we will see teams use these starter kits as the starting point for their application development.  The use cases here show how Watson services can be combined and use together to solve some common patterns of cognitive computing capability.

Keep in mind that nobody is going to become an expert with Watson and cognitive computing overnight, but these resources can help you effectively use your time to learn about WHAT is possible, and HOW to do it.  They will give you a great start on your journey to becoming a cognitive computing expert.