This is the second in a series of blogs about how to assess and possibly improve the performance of your Jazz based development tools. In Part 1 – Is My Network Causing Me Pain, we looked at how to assess the relative health of your supporting network infrastructure. Today we are going to focus on the use of the Jazz Administration console.
Tour of the Jazz Administration Console
The Jazz Administration Console is available only to users with Jazz Admin rights. You will need to have these rights to access all of the various pieces of the Jazz Admin console. When you get onto the console, you should see something that looks a bit like this:
Since we are focused on troubleshooting Jazz Performance, we’ll first focus on the performance of the Jazz Team Server. Click on the link for the Jazz Team Server (the first in the list), and you should see something like the following:
There is a lot of information here, so let’s look at what some of it really means. You will see some basic health statistics in the upper left, indicating how long your server has been up, and what it’s public URI is. Further down you see some basic information on the Server VM. In some instances it can be important to be aware of what JVM is running.
On the upper right, you see some statistics on memory usage. Do not be fooled by the free memory percentage. That figure indicates the amount of free memory left from what is already allocated. In this case we have 4% free of the 244MB currently allocated, with a maximum of 1200MB. So we have plenty of room to grow. Jazz is a lot like any other Enterprise application, they love to consume heap memory and cache as much as possible to help improve performance. We will often suggest 4GB of heap for most Jazz applications, although your installation may need to be higher or lower depending on your user load. Just don’t jump to conclusions based on these numbers.
Now let’s look at some of the tabs that are shown on the left hand side. First we’ll take a look at the Statistics for your server. Click on the Statistics tab on the left, and you will see the Server Statistics displayed.
Now this looks more like internal kinds of data. This page shows you some historical data on the Jazz services being used. The top group shows the Web services with the longest average execution time. You will notice that for my small installation, grabbing the license seems to be the thing that takes the longest. Monitor these values, and keep in mind that GET operations are typically OSLC fetches of data. If you have get operations that are taking a long time (say more than 3 or 4 seconds), then check with the Jazz team to see if this is expected.
The second grouping looks at the asynchronous tasks in the same way. Keep an eye on what your values look like, and see which tasks seem to be taking the longest to complete. Sometimes this can give you an idea of the types of operations that are most costly, in terms of response time and performance.
The remainder of the sections look at the frequency of usage of these services (the Top Count statistics), and some caches hit statistics. In general the cache hit statistics are not that important. The Top Count statistics, when coupled with the average execution times, can show you where most of the processing in your Jazz instance is taking place.
Now let’s move on to some other interesting areas of the Jazz Administration Console. Click on Component Status on the left hand nav bar. You will see a status dialog come up for all of your various Jazz server components. All of the decorations should be small green checkmarks. If you see small red decorations, that means that some of your services are not running.
Now we’ll move on to the next display. Click on Active Services on the left hand nav bar. Now you will see a series of grey bars with the names of the currently active services listed below them.
Clicking on the small triangle on the left side of any one of these bars will then display the call stack of that particular service. The key thing to look for in this display is the running time of each service. Look for services that have long running times, this may indicate services which are unable to proceed due to a lack of CPU cycles, repository access, memory, or some other reason. Get familiar with the services that seem to be most active in your environment, since it will help you understand where most of the processing time in your environment is being spent.
Note that this display is just a “picture in time”. It is static, and shows you the active services at the time you brought up the display. Hit the refresh button on your browser to update the active services and see what is happening at this present moment.
The next section of dialogs in the left hand nav bar deal with Configuration. These displays allow you to see how you currently have your Jazz server configured, and allow you to easily change some of these settings. You can look through these on your own, they are self explanatory.
The one that you should be aware of is the Advanced Properties display. In this display you can see all of the configuration settings for your Jazz server. BE CAREFUL. It is easy to change some of these settings and completely screw up the performance and the operation of your Jazz infrastructure. Do not make ANY adjustments to these settings unless you are an experienced Jazz Administrator, or you check with Rational support first. Remember to always keep a record of the changes that you make, but you’re smart and you already knew that.
The next area of interest in the Jazz Administration Console is the section on Licensing. On the left hand nav bar, click on License Key Management, to bring up the license key management screen. Take a look at this display, so you know where you do any licensce key management operations. Also check out the Issued Leases display, which will show you who is currently using any floating licenses that you have.
The key thing to check in this section is the License Usage display, just select the middle entry in the Licensing section. The License usage display will produce a report of floating license usage on your Jazz server over time. You just need to select the range of dates that you want to report on. You can use this for a couple of different purposes when you are looking at the performance of your Jazz server. It can help you determine when license usage is ramping up, and allows you to alert your team that they may need to acquire additional licenses. It can also be used to get a rough estimate of your user load at different points in time. If your performance issues seem to align with periods of high license usage, then you may be experiencing a user load issue on your Jazz server. It’s another piece of information that you can use to help you understand what is going on in your Jazz environment.
The final section has displays which show the Jazz Administrator the various “friends” and relationships that the Jazz server is aware of. These can be useful when debugging issues related to the connectivity between various Jazz applications.
Final Thoughts
One thing to keep in mind when looking at the Jazz Administration Console is that each of your Jazz applications has it’s own console. Remember that the Jazz team Server (JTS) is a separate application from your RTC server, which is distinct from your RQM server, and so on. You can find information to help you on the Administration console of each of these servers.
What Next?
This series of articles is continued. Here is a list of the topics covered:
- Jazz Performance Part 1 – Is my Network Causing me Pain?
- Jazz Performance Part 2 – Using the Admin Console
- Jazz Performance Part 3 – What does nmon have to do with my Jazz Server?
- Jazz Performance Part 4 – Why WAIT, see what your Jazz Server is doing
- Jazz Performance Part 5 – Keep an http watch on your Jazz Server
- Jazz Performance Part 6 – Obeying the Laws of Physics
- Jazz Performance Part 7 – What is JazzMon and Why Should I Care?
Dan Toczala's Blog 
Dan,
I am gathering that the “stats”link was removed from the admin page?
under the status section all I see summary , diagnostics active services.