Jazz – Better the Second Time Around?

The Situation

I have just spent a couple of days with a customer that is looking at Jazz, and Rational Team Concert in particular, in a proof of concept.  The customer is great.  They are intelligent, fun to work with, and they have a good understanding of what they want to do, and how they want to get there.  There is only one problem.

They set up their proof of concept environment on “borrowed” hardware from their infrastructure team.  At the end of last week, the infrastructure team reclaimed the database server, but never asked the team if they were still using it.  So my customer came in one morning to find that nothing worked, and after some quick troubleshooting they discovered that their database was gone.  Two weeks of work on a project was gone, in a single instance of non-communication.  A bad day.

Then they discovered that their database had never been backed up.  A REAL bad day.

So now we have to recreate the proof of concept environment, and quickly.

What Happened

Since we had a copy of their process template, and copies of their build.xml scripts, we have made significant progress in restoring their environment in less than two days.  This was pretty easy to do, and didn’t require much effort from the team, just myself, their project manager, and their lead architect.

We have been able to restore (and improve upon) their work items, their workflows, and the governance of their process.  Their architect has been able to recreate their build environment, as well as the automated testing and code coverage pieces.  We are now importing the code from their original source code repository, ClearCase, and re-establishing their synchronization with ClearCase (which is being utilized by the rest of the organization).  We still have some issues around publishing and saving code coverage results, but in two days we have essentially recreated their entire development environment.

What We Relearned

I really like Jazz, but this highlighted a couple of things for me.  The most important one is that when doing a proof of concept, it is vital that everyone know what is going on.  It’s OK to do these proofs of concept on borrowed hardware, but make sure that your organization is aware of how long you will need these resources.

The next lesson is the same one I relearn every time I kill the hard drive on my laptop.  Backups make sense.

What I Really Learned

Some of the more subtle things that I learned have to do with the ability of people to adopt the Jazz technology.  This team had just lost 2 weeks worth of work, which is not a trivial loss.  It would have been easy to just pitch the proof of concept overboard, blame the tool,  and continue working the way that they always have.  However, this team liked the Jazz environment enough to shrug off what could have been a substantial blow to the project, and continue on with Jazz.  They even used this as an opportunity to further refine the workflows that they had previously had in place.  This was a powerful message to me.

The second thing that really struck me was the relative speed with which we were able to redeploy the Jazz environment.  We has just lost all of our server setup, customizations, all of our work on the process, the project organization, and our work item customizations.  If this had happened to us with an integrated set of tools, the work to redeploy and reconnect everything would have taken us a week.  This was the significant nugget of wisdom that I was able to pull out of what could have been an unfortunate situation.

Jazz in the Real World

First Blush

In working with customers, I find many different attitudes and expectations associated with the Jazz platform, and the philosophy behind Jazz.  People love the idea of a unified software development platform, and think that the technology is fantastic.  They see the power of having a unified layer that tools plug into, like Eclipse, and communicate through.  They like the portability of the solution, and the web/WAN enabled clients.  They like the collaborative possibilities of the Jazz platform.  They like the ease of the integrations.

But they miss the big story.

What Jazz does is far more profound, and far more subversive (pardon the open source pun).  The organizations that see the strategic possibilities of the platform, see some of the following things.

Visibility + Transparency = Accountability

One of the most compelling things about the Jazz platform is the thing that nobody talks about.  It has to do with the visibility and the transparency that the platform provides.  Since all development artifacts are related to each other within a single repository, and since measurements can be taken from these artifacts, information on a project can be easily determined.

This can be done regardless of the role an individual plays on a team.  Software developers can see what tasks and stories are unassigned, and select the ones that they can best address.  They can collaborate with other developers on tasks.  They can easily query the repository to look at file histories, or to recall other similar changes made.  They can see the metrics associated with the work items that they have delivered, like unit test results, code coverage, and static analysis metrics.

Project managers can see all of this same information, and can easily track the team as a whole.  Results and metrics roll up to the team level, and the workload of team members can be adjusted in real time.  The overall progress of a project can be easily determined, and adjustments can be made.  Since metrics are based on the objective status of the actual development artifacts, no time is wasted in collecting status from team members.

Executives also benefit from the transparency and visibility provided.  Executives can easily see how their teams are doing, at a high level.  Current status and results can be viewed in a series of dashboards which are configurable, allowing executives to look at the things that interest them.  Since all of the metrics are based on the states and contents of the software development artifacts, executives can be confident that the data that they are seeing accurately reflects reality.

What are the implications of this?

All of this visibility to team activities and progress  imply a degree of accountability.  Since everyone can see what your team is doing, and what progress your team has made, and what you team has committed to.  Your progress and your work are there for all to see.  This accountability makes people think and work more efficiently, since there are no organizational “walls” to hide behind.  That is what organizations want, they want teams to be accountable.  With a Jazz based software development infrastructure, teams are accountable.

Bear in mind that this accountability is not one way, and it doesn’t stop at your immediate superior.  The transparency and visibility supported by a Jazz based environment means accountability up and down the entire software development organization.  Think about that for a second.  Do you have true accountability through the length and breadth of your organization?  Probably not.  It is a cultural change that could have a significant impact on how software is developed, and on how organizations value their software development teams.

Freedom of Choice

The Jazz platform is built along open source types of principles.  The source code can be downloaded.  The API is open and published (an “official” API will not be available until the Summer 2009, it is still undergoing change).  In essence, the philosophy of the Jazz platform is to provide a platform that will encourage the development of new tools and technologies that will help make software development more efficient, cost effetive, and valuable.

Is it open source?  No.  It is Open Commercial development.  Will IBM be selling products that are based on Jazz?  You bet.  IBM loves revenue as much as anyone else.  Will other vendors be providing tools on Jazz?  They already are, a series of business partners and independent organizations are busy making things “Jazz enabled”.  Wil open source tools work with Jazz?  Some already do (Subversion), with more organizations forming bridges and connectors from open souce tools to Jazz.

The wonderful thing about Jazz is that the platform is extensible and, like Eclipse, can support the creation of plugins that provide additional functionality.  The Jazz architecture provides access to the artifacts, but the definition of the data is tool independent.  So if two teams are using the Jazz platform, one might be using Subversion (via Subeclipse), while the other might be using the source control provided by Rational Team Concert.  Jazz works fine with both, and from a high level view, the tools being used have no impact on the metrics and reports provided. 

People I have discussed this with seem to understand this, and appreciate that it is a good thing.  The real missing piece are the implications and strategic advantages that this provides.

What this freedom of choice allows an organization is quite significant for a number of reasons.

For most organizations, imposing a standard set of tools across the entire organization is not realistic.  Now an organization can make a set of similar tools available to teams, and allow each team to choose the best tool for their particular job.  Purchase of tools can now be based on the value that the tool brings to the individual project, and organizations can stop trying to get everyone to use the same tools in the same ways.  Think of all of the turf wars that were based on tool choice that you have witnessed in an organization.  How much time and energy were wasted?  Quit fighting internally, and focus your energy on productive work.

Today, tools rapidly become obsolete.  Yesterday’s standard tool is now an antiquated relic, and development teams clamor to get onto the newest, most fully functional tools.  Many tools have become commoditized.  Remember when you used to get a bill for having an email address set up with a couple megabytes of space?  How many email addresses do you have today?  (“Let me see…..one for my job, one for home, one for my professional development, one I give to websites that will send me spam,……”)

A Jazz based infrastructure allows each individual project to assess it’s own tool needs, as well as their risk tolerance (because that new tool isn’t really stable yet), and select the tool that best fits it;s needs.  Since all of the data associated with the software development effort is being managed by the Jazz infrastructure, the tool is now just a means to an end.

A good analogy is a building contractor.  He has nail guns, hammers, and sledgehammers.  All of these tools do the same thing, but each tool has different strengths and weaknesses.  Do you want to be doing demolition with a hammer?  Do you want to drive those finishing nails with a sledgehammer?  Choose the appropriate tool for the job being done.

Reality of Migration

As I mentioned above, a Jazz based infrastructure allows each individual project to assess it’s own tool needs.  Some organizations may standardize on a specific tool as a way to cut purchasing costs, adminstrative overhead, or to provide a common basis for communication within the organization.  The most painful part of this is the migration of the organization to the standard tool.  Not all projects can migrate at the same time, and some have a large number of artifacts that need to be migrated.  

The cost associated with migrations alone will usually far outstrip the cost of purchasing a commercial tool.  This is true for migration to any new tool (commercial or open source).  the movement and reformatting of data, training of users, and establishment of a support mechanism will usually be the most costly part of any new tool, often by a factor of two or more.  The amount of time and effort expended on these activities can be significant as well.

Once again, Jazz provides an answer.  Depending on the tools currently being used by the projects, import tools and connectors can be used to make the existing data visible and available to the Jazz infrastructure.  Projects currently using Subversion can continue to do so, if that is what they would like to do.  Migration activities can be planned for convienient periods of time, or can be avoided completely.  The cost savings of avioding migration, while still being able to provide additional capabilities to team, is significant.  The opportunity cost of avoiding this work, and the management of these activities, is harder to quantify, but no less significant.

As new tools, or new tool releases, come out, teams can implement the tools without the costly and time consuming effort of data migration.  Since all of the data lives within the Jazz infrastructure, and the only thing that needs to be done is the deployment of the tool itself.  This is true of ANY Jazz based tool, regardless of its origin (open source, IBM, or some third party).

Closing Thoughts

Three of the biggest benefits of a Jazz based software development infrastructure are things that may not be apparent when first looking at the Jazz technology.  Jazz forces an organization to be accountable.  The increased visibility and tranparency of Jazz supported projects combine to impose an implicit level of accountability on software development teams.

Jazz gives projects and stakeholders a freedom of choice in their selection of the tools that they want to use.  By having a common framework, REST enabled interfaces, and a focus on keeping the data decoupled from the tools, Jazz allows an organization to govern it’ssoftware development activities, without imposing unrealistic tool standards.

Finally, Jazz addresses the realities of migration, and recognizes that migration activities are often the most expensive portion of any new tool/technology rollout.  By easing the burden of migration, and with the data being decoupled from the tools, Jazz allows for more rapid and cost effective migrations now and in the future.