Translating from ClearCase/ClearQuest to RTC and Jazz

Update on September 10th, 2010

NOTE: This information is now posted, in a slightly different format, on Jazz.net.  The section comparing ClearQuest and Jazz can be found as Comparing concepts between ClearQuest and RTC, and the section comparing ClearCase and Jazz can be found as Comparing concepts between ClearCase and RTC.  These Jazz.net articles have been updated, and the quality of information in them is better than what is here.  I leave the original content here for those of you who are too lazy to get a Jazz.net id and click on the links above (grin).

The original blog posting is below.

I recently spent time in Korea helping our customers understand Jazz Extensions and OSLC. Many of the questions came from people who are very familiar with Rational ClearCase and ClearQuest. As we discussed Jazz and Rational Team Concert, they kept trying to tie the Rational Team Concert concepts back to the concepts and terminology that they already knew. So, I thought it would make for a good blog post, if only to buy myself some time to learn more about OSLC and the Jazz API.

WARNING: This is a LONG blog post, but if you’re familiar with ClearCase and ClearQuest, it should help you better understand Rational Team Concert concepts and terminology. Continue reading if this interests you!

Comparing ClearQuest and Jazz

Let’s begin with some basic terminology and comparisons from ClearQuest. In ClearQuest you have the concept of a Record Type. These Record Types determine the different types of records that you can have in ClearQuest. They can be either stateless, or stateful records. Each Record Type has a number of Fields, and these Fields can be of different basic types (like enumerated, text, numeric, etc.). The Fields can be mandatory or optional, and they are often used to help categorize work. They can also be linked to each other. The Stateful Records are used to model process flow through a development organization. All of these various Record Types, and the enumerated data used to provided values for choice lists, are captured in what is commonly referred to as a ClearQuest Schema. These CQ Schemas can be exported and imported, but the format of the information is specialized and proprietary. As users interact with the system, they create CQ Records, which are instances of the type with data specific to some change request or work to be performed.

In Jazz/Rational Team Concert (RTC) we have a Process Template which is used to dictate the structure and operation of RTC. In Jazz/RTC we have Work Item Types, which are roughly equivalent to the Record Types in ClearQuest. A Work Item always has a state model associated with it, there is no concept of a stateless type. Jazz Work Item Types have Attributes, which are roughly equivalent to the Fields in ClearQuest. These are all identified in the process used by a project. When a project in Jazz/RTC is started, it uses a Process Template to define the Work Item Types. It defines their Workflow and state models, their Attributes, and the various process operations that act on these Work Items (more on that in a little bit). This is all done with a “point and click” type of user interface. The selections and configurations done in this interface are captured in an XML representation of the process. This can then be used to create new Process Templates for other projects. As users interact with the system, they create Work Items, which are instances of the type with data specific to some change request or work to be performed.

Within ClearQuest, the Workflow can be characterized as a series of states. Individual records transition between these via a series of actions. In RTC, the concepts of states and actions are the same, with actions being used to transition between the various defined states. This is where RTC and ClearQuest begin to diverge a little bit. ClearQuest allows users to see the various stored records (data) as reports, query results, or diagrams fed by query results. These report formats may be native, delivered with the product, but are often customized and defined with Crystal Reports. Jazz/RTC allows users to see the various stored work items (data) as reports, query results, or diagrams fed by query results. The report formats may be native, delivered with the product, but are customized and defined using BIRT. It also exposes this data in views called Plans. Plans show the Work Items (data) in a format that can be used for planning and monitoring progress of sprints or iterations. The individual Plans can be thought of as windows on the same data that just display it in a different way. The difference between reports and graphs is another example of looking at the same data, in different formats.

ClearQuest maintains a history of CQ Record transactions, and most customers will implement an Audit Trail capability with ClearQuest. Historical data and historical reports are built using data from the individual record histories. In Jazz, Work Items have their own history, with comprehensive tracking of field and state changes. One additional thing that Jazz will do is to populate it’s internal data warehouse over time, collecting and accumulating data from the Work Items. This data warehouse is used to provide data for all of the time-oriented reports and graphs shown in Jazz. So, when looking at reports on data over a period of time (a burndown chart, for example), the data that you are seeing is coming from the Jazz data warehouse. This pretty much covers the differences between Jazz/RTC and ClearQuest from a basic functionality point of view.

So what about customization and business logic in the process? When using ClearQuest, customers will implement more involved business logic by writing “Hook Code” in either Perl or Visual Basic, with Perl being the more popular choice with the most examples. This Hook Code is tied to particular events that occur to either a CQ Record, or one of it’s Fields. There is a rich set of API calls that will allow a schema designer the ability to perform various operations on those CQ Records and Fields. In Jazz/RTC, the ability to modify the behavior of Work Items and Attributes is based on the built in security model (which is also customizable), and a series of common operations that customers will often perform. These are exposed as a series of operational behaviors and switches that the Process Template author can modify. For those things that are not already available in the tool, Process Template authors can write Extensions in any language, with Java being the popular choice with the most examples. There is a rich API and SDK that can be interacted with, and used to manipulate Work Items and their Attributes, as well as returning user guidance and error messages.

So how about a quick look at what you have learned thus far:

ClearQuest Concept	Rough Jazz Equivalent	What it does
CQ Schema	Process Template	It defines the process and how the tool exposes the process
Workflow	Workflow	A state model that defines the flow and transition of work products
CQ Record Tyoe	Work item type	Defines an object type for holding data, with it’s own state model, defined fields, and presentation
CQ Record	Work item	An instance of the record type/work item type, with it’s own unique data
CQ Field	Attribute	One of the data entry fields, part of a record/work item
Hook Code	Extensions	Code associated with more customized implementation
CQ Database	Repository	Where information is stored
n/a	Plans	A different display or visualization of the current work item data, useful in project planning

Comparing ClearCase and Jazz

Now let’s take a look at the differences between ClearCase and Jazz/RTC. ClearCase is a configuration management system. Customers may use it stand alone, or they may use the UCM functionality. UCM forces users to create software Components, and then creates Baselines on those components. Files being modified are Checked Out, modified, and then Checked In within the context of a View, which is a work area for development. Change Sets are built up with each of the individual changes made as part of an Activity. These Change Sets are then shared with other team members through the Delivery of those Change Sets. ClearCase stores it’s artifacts in a proprietary format, and stores all of it’s data in repositories called VOBs. Each VOB may contain one or more UCM Components. The individual VOBs can store artifacts in any directory structure that a user would like to use. ClearCase supports parallel development, and when there are conflicts between versions of the same file, these are resolved using Merges. In base ClearCase, VOBs may contain Branches, which are logical “splits” of the code to enable parallel development activities to take place in isolation. In UCM these Branches have some more formal constraints, and are referred to as UCM Streams. So a team member will deliver his or her changes from their own working area, called a View, to a Stream. ClearCase has a concept of a Dynamic View, which is a workspace that changes without any user intervention, based on user configured settings. ClearCase also has the concept of a Snapshot View, which is a copy based workspace, associated with the state of the repository at a point in time. Snapshot Views are brought “current” by Updating the view. From either view type, Baselines and Change Sets may be Delivered from Stream to Stream, with any individual file Merges being detected by the system, and then resolved by the user. As Components get Change Sets applied to them, the individual Components will have Baselines (essentially a label across all artifacts within the Component or VOB) applied to them. A “master baseline” with a collection of Baselines across multiple Components is called a Composite Baseline.

In Jazz SCM there are many of the same concepts, but there are some subtle changes. Jazz has a single repository, which stores its artifacts as REST-enabled resources. These resources are stored in a database. Jazz supports the use of many enterprise database technologies, including Oracle, DB2, and SQL Server. This is commonly referred to as the Jazz Repository. In Jazz, software is broken into software Components. These Components may contain artifacts in any type of directory structure, and parallel development is supported by use of Jazz Streams. Users working on Jazz Components will join a project, and will create their own Repository Workspace (which holds copies of all of their Change Sets), as well as a personal Sandbox work area on their local machine. As files are modified, the Jazz client will track those changes, and the user will Check In their changes. There is no concept of check out, although Jazz does support resource Locking. When a user sees changes in his or her Sandbox, they will Check In the changes, and relate them to a Change Set, which collects the individual changes and stores them in the Repository Workspace area. These Change Sets can then be Delivered to one or more Jazz Streams. The user Sandbox, or Eclipse Workspace, can have its status checked at any time, with the interface indicating all potential outgoing Change Sets (work the user wants to Deliver), as well as incoming Change Sets (work done by other team members). Users are able to select Change Sets to either Deliver (sharing their Change Sets with the team), or Accept (applying Change Sets in the Stream to their own Repository Workspace and Sandbox), on an individual basis. Baselines of the Components can also be created, and a collection of Baselines across multiple Components is called a Snapshot. This should not be confused with a Snapshot View in ClearCase, as the concepts are completely different.
There are a number of subtle differences in the way that these systems work, and it gets reflected in how users will interact with these systems. In ClearCase, users are often encouraged to create Streams and Branches to isolate their work. Often customers will create multiple Views (workspaces), with one set of changes being done in each View. This is done so changes can be isolated from each other, and to ensure that no artificial dependencies are created between Change Sets. In Jazz, users are encouraged to perform one change at a time within their Sandbox (workspace). Users working on multiple changes have a choice, they will either create multiple Sandbox and Repository Workspaces, or they will Suspend work on one Change Set, before beginning on another one. This ability to Suspend work essentially takes all of the changes associated with a Change Set, and stores them in the Repository, so a user can Resume (or reapply) these changes at a later time. The Workspace is then returned to the state that it was in prior to the changes from the Change Set being applied. This helps cut down on the number of Workspaces that developers must keep track of.

When using ClearCase with ClearQuest and UCM, it is possible to associate a ClearQuest Record with a Change Set. This requires some configuration, and availability of both the ClearCase repositories (VOBs), and the CQ database. In Jazz/RTC, all of this information is in the same Repository, so a link is created between the Work Item and the Change Set. Since everything is REST-based, many different types of Jazz objects can be linked to each other, providing greater flexibility in the relationships between development artifacts.

ClearCase Concept	Rough Jazz Equivalent	What it does
Check Out/Check In	Check In	Commits changes to the repository
Activity	Change Set	Collects a set of file changes into a single package of changes, usually associated with a CQ Record/Work Item
VOB	Jazz Repository	Stores data under source control – single Jazz repository, multiple VOBs
Dynamic View	n/a	A virtual workspace whose contents are dynamic
View Storage	Repository Workspace	There isn’t a good equivalent for the repository workspace, but a view storage area on a view server comes close
Snapshot View	Sandbox/Eclipse Workspace	Copy-based workspace used for development activities
Updating	Sychronizing/Accept	Update of the user workspace, where changes from other team members are brought into the user’s personal work area.
Deliver	Deliver	The promotion of a change set from the workspace to the stream, where it is shared with the larger team
Reserved Check Out	Locked	Ability to “reserve” a resource, making sure that you are the only user who can modify the resource
Baseline	Baseline	A collection of files (configuration) within a component at some point in time
Composite Baseline	Snapshot	A collection of different components, and specific baselines from those components
UCM	Process Templates (like Scrum)	An out of the box process that defines record types/work items, and the relationships and process associated with them
Stream/Branch	Stream	A logical “split” of the code to enable parallel development activities to take place in isolation
Merge	Merge	Taking versions of the same file on two different branches, and reconciling the changes made to those files
n/a	Suspend/Resume	Ability to temporarily store or restore changes made in a workspace, to return the workspace to some known configuration

Ok, so now that you have some background on what the differences between ClearCase/ClearQuest and Jazz/RTC are, and you have a mapping of the concepts and terminology used by the tools, let’s look at some more of the real world implications.

When customers need wide geographic availability of their data, there are things to consider. Since Jazz is REST-based, and works via http, it can be deployed in a data center and accessed from anywhere in the world. Jazz also allows for the configuration of proxy servers to help speed performance for those systems that have a high load of source code and development data. With ClearCase and ClearQuest, data is replicated between different physical sites, so users can experience “local” performance. This is called MultiSite. This results in some changes to the way users do their work, since replicated data may be read/write at only one location, and read only at the other locations. This is often addressed by creating Branches/Streams for each physical location.

Often long time users of ClearQuest technology have created collections of stateless records, which are used to determine drop down lists presented to the user. These are often called dependent choice lists. Jazz/RTC does not handle dependent choice lists well, since there is no concept of a stateless record in Jazz/RTC. Currently there are techniques for building dynamic/dependent choice lists in Jazz/RTC, but these are based on more static data, and not on existing field values.

There are other differences between the two sets of tools, and I encourage you to add and share your own observations and comments on these differences. When evaluating which tools to use, you should have an understanding of the concepts and basic terminology in use. Choose tools based on information and your situation, not on guesswork and impression. Educate yourself, and reach out to others in the community to help explain the benefits and strengths of each tools being evaluated. Jazz is a fantastic tool, but there are situations where it’s current capabilities may not fully support what your organization needs. Many of the customers that I work with have environments with a mix of many different tools that do the same thing. The most common of these are organizations that use ClearCase, Jazz SCM, and Subversion. Utilizing bridge and synchronization technology, all of these tools are able to exist, share data, and integrate in a Jazz based environment.