Using Scrum Methods with IBM DevOps Services

Note: I co-authored this with Patchanee Petprayoon, so you can also find this on her WordPress blog, Ppatchanee.

This article is in response to a number of frequently asked questions about sprint planning and scrum methods, as well as some links to other resources we have created or found to answer those requests.  Before we dive deeper into the topic, let’s take a quick look at what type of scrum work items are provided in IBM DevOps Services. If you have never used any scrum methods before, this can be overwhelming at first.  That’s OK, just read through this and try to digest as much as you can.  If you are familiar with Scrum and Agile methodologies, just skim through all of the explanations to get to the meat on how to do things with IDS.

We call out some sections of this whole process in this indented green text.  These represent best practices, or areas where you can dive more deeply into how IDS works. 

You might also note that a lot of the links for terminology in this article lead to Wikipedia references.  It’s not because we love wikipedia, it’s because too many of the good articles out there have a lot of product references (like this one does – with IDS).  I went looking for some content that was tool agnostic – and Wikipedia is often the best bet for that type of material.

Before We Begin

If you haven’t already registered for IBM DevOps Services (IDS) and Bluemix, you should register now.  It will help you follow along with our examples and will allow you to try a few things for yourself.  You can use IDS alone, use Bluemix without IDS, or use them together.  You may have used the Scrum Process with Rational Team Concert in the past.  If you did, then a lot of this will look quite familiar.

Getting Started with IBM DevOps Services (IDS)

Once you have registered with IDS, you will want to create a project.  To do this, go out to your IDS dashboard.  Click on Create a Project.  You will see a new project dialog come up.  Give your project a name (like “Sample Scrum Project”), something that will identify it.  Then select where your source code repository will be.  For the purposes of this article, it doesn’t really matter, since we’ll just be looking at some common Scrum methods, and will not be dealing with code.  Then make sure that you have a checkmark in the box for “Add features for Scrum development”.  If you are an adventurer, and want to play around with some code, and then deploy it to Bluemix, you might also want to check the box for “Make this a Bluemix project”.  Finally, hit the “Create” button in the lower right, which will create your new scrum project.

So now we have a Scrum project created, and you can see it in the browser.  Now we want to set it up correctly.

In the future, we’ll want to add other members of our team to this project.  To do this, just click on the link for “Members” on the left hand nav bar, and then click on the “Invite Members” icon to add members of your team to the project.

Right now, we just want to learn how to use IDS in conjunction with some common Scrum practices.  One of the first things you’ll need to do is to decide on the length of time of each of you iterations.  In Scrum, these are called Sprints.  Each Sprint represents a period of time where the development team will develop and test code, and commits to have an executable application at the conclusion of the sprint.  That means that you have a running application at the end of each sprint.  It may not expose all of the functionality that has been developed, and some functionality may not be present, but it will execute.

Most scrum teams run with 2 week sprints.  To set up your sprints, start on the overview screen, and press the button labelled “Track & Plan”.  Once on this screen, select “Sprint Planning” from the nav bar on the left.  Since you have just started your project, you don’t have any sprints right now.  So let’s create some.  Click on the “Add Sprints” button in the Sprint Planning window, and add 4 new sprints, and make them 2 weeks in length.

If you click on “Edit Sprints” at this point, you can see how you can now go and add new sprints, rename your sprints, and change the dates on your sprints.  When working with IDS, we suggest naming your Sprints the same as the end date of the sprint.  That way your stakeholders who are unfamiliar with Agile can look and see when their user stories will be implemented at a glance.  They really don’t care about sprints or Agile development, they want to know when they can expect things.

Defining Your Initial Work

Now you have the basic project setup done.  It’s now time to start thinking about doing actual work.  The first thing to understand is the different types of work items available to you to capture your work.

Work Item Type What it is used for
 Story (sometimes called User Story) A user story is a description of some functionality needed, told from the perspective of the end user of the application.  Some people find this equivalent to a use-case.
 Task Tasks are single user tasks used to accomplish some well defined and well scoped effort.  Stories are typically broken down into the multiple tasks, representing the work needed to accomplish the implementation of a Story.
 Epic Epics are stories that are too large to be contained within a single sprint.  Typically an Epic can be broken down into two or more Stories.
 Defect Defects are bugs, problems with the application.  Many Scrum teams will treat them similar to how they treat stories, by decomposing them into the tasks needed to be done to fix a particular defect.
 Impediment These are the risks, issues, and environmental factors that stop work from being accomplished.  Typical impediments may include things like waiting for properly signed SSL certificates, waiting on hardware availability, lack of stakeholder availability, and other similar situations.
 Adoption Item Adoption items indicate when changes made by one team need to be accepted (and integrated) with another team.  An upgrade to a new database version is an example of this.
 Retrospective Retrospectives are held at the end of each sprint, and they allow the team to reflect on what is working well, and what isn’t working well.  It is an attempt to keep the team focused on continuously improving their effectiveness.  Retrospective work items capture the notes and observations made by the team during a retrospective.
 Track Build Item Used to track builds, and relate them to them functionality that they will deliver.  Not a lot of teams use these.

So at this point you should begin creating user stories, epics, impediments, and adoption items for all of the work that you are aware of.  Just click on the “+” in the backlog box to begin creating your first work items.  Once you click on it, just type in a brief headline or general description of the work.  Don’t worry, you will be able to fill in more detail later.  Click on the small icons below the text, to change the work item type, add a more detailed description, and so on.  As you add work items, they will begin to display in your team backlog.

First work item
The edit sprints button, some text for a first work item, and the icons to change the work item attributes

Backlogs are a critical part of scrum development.  Your backlog contains all of the work that your team thinks that it may need to do, but that the team has not yet committed to doing.  If you haven’t scheduled it, and assigned the work to a particular sprint, then it should be in the backlog.  Mature products may have backlogs with 100 or more items in them.  We’ll discuss how to manage your backlog in the next section.

After you have created your initial work items, take a look at your backlog.  It should show all of your work items.  Click on any individual work item, and you will see the details about that work item.  Go back to your sprint planning view, and you will see the work items displayed along with some summary information.

When looking at the details of a work item, it is important to note the fields in the work item.  All of them are self-explanatory, but they do impact how your team does it’s work.  Often lists that are displayed (like your backlog) will have higher priority items at the top.  The discussion field is used to discuss work items, and to provide a running status on the work items. 

When you want to ask someone something in a discussion, click on the link icon with a person, and you will get a dialog where you can choose the specific person that you want to identify.  Now once you save your changes, that person will get an email notifying them that you have asked them a question, along with a link to the workitem.

If you have people always asking about the status of some work, just go to the links tab on a detailed work item, and add that person as a subscriber.  You can add yourself or other people as subscribers to a work item.  whenever a work item is modified, all subscribers get an email that notifies that of the modifications made to the work item, as well as a link to the work item.  We end up using much less email, and having far fewer status meetings, because by using subscribers we are able to keep everyone informed on the progress of critical work items.

Backlog Management

Let’s face it; even though we try our best to estimate things, we’re often very wrong.  We’re not wrong because we are dumb, we’re wrong because we don’t have enough information.  So when using Scrum, we look at each story on our backlog, and we estimate the size/complexity of the story using story points.  Teams will use various methods for estimating story points (some use Planning Poker).  The way that we size the story is very easy and straightforward. If you think that the user story can take a short amount of time to complete, then you give it a low number. If you think it’s going to take a lot of effort to get the user story done, then you give it a high number.  As time goes on, you will get a better feel for estimating work in this way.  You can do all of this work while in that same sprint planning view, capturing your decisions and discussions in the work items as you go along.

There is heated debate about assigning story points to user stories.  Some people think that it is a measure of risk and complexity, some think it is a measure of difficulty, some think of it as a measure of effort.  Do what works for you.  I like Michael Tang’s quick view of story points, but you can find other viewpoints from Mike Cohn and others.  Just spend 30 minutes Googling this and educate yourself.

Once you have estimated the relative sizes of your stories, you are now ready to do some backlog grooming.  In the left hand nav bar, you will notice a link called “Backlog”.  Click on that link, and you should see a nice list of all of your stories.

BacklogGrooming
Backlog Grooming Example

Now we can begin grooming this backlog.  Drag the most important story, the most critical one, to the top of the list.  Now drag the second most important story, and drop it below the first story.  Keep doing this until you have a ranked list of your user stories.

Clicking on a user story will bring up details about it, which you can use when discussing them.  Be careful when dragging and dropping user stories.  If you drop a user story on top of another user story, it will make that user story a child of the user story it was dropped on.  Sometimes you want to do this.  If you didn’t want to do this, then click on the small plus sign “+” on the left of the user story, which will display it’s children.  Then click the small “x” on the left of the child user story, to remove it from being a child.  If you drop it in-between two user stories, it will get ranked between them.

This is your ranked backlog.  This is a critical step for your team, because this indicates the relative priorities of the work that you have been asked to do.  When we get into sprint planning in the next section, you will see how we look at the top of the backlog when planning our next sprint.

Backlog grooming is an important and ongoing activity.  Some scrum teams will do backlog grooming once per sprint, others will do it monthly, and still others will try to do it on an ongoing basis.  The thing to remember is that stories on the backlog need to be reviewed periodically, to ensure that the estimates are accurate, and that the business and development environment have not changed and impacted their priority in relation to other stories on the backlog.

Sprint Planning

Now we get to the key piece of software delivery planning, Sprint Planning.  In your IDS browser, in the left hand nav bar, click on the link called “Sprint Planning”.  When you do this, on the left hand side you will see the ranked list of your backlog items, with the highest raked items on top (I told you that backlog grooming was important!).  On the right hand side you will see an empty list, which represents your first sprint.  Now at this point, you would begin to drag user stories and work items from the backlog (on the left), to your sprint (on the right).  Go ahead and try it.  Move the top ranked user story from the backlog to your sprint.

When you do this, you will notice some changes.  Now in the right hand list, click on the link for “Team Progress”.  What should you see?  Well you should see that the story is now scheduled to be done during the first sprint.  You’ll also notice that the story points assigned to this story show up in the team progress (you now show as having completed 0 out of x story points).  Observant users will also see that the ranked list has updated, and that the next item on the backlog now has a ranking of 1.

Continue to assign stories to your first sprint for what your team can GUARANTEE that they will finish in the timeframe of the first sprint.  That means COMPLETELY FUNCTIONAL software that satisfies the user stories.  Do not over-promise – this is your team’s commitment to complete the work in the timeframe of the first sprint.  Keep in mind that you’ll also have to do testing, and that some bugs may pop up and need to be done as part of the work of this sprint.

The amount of story points that a team can normally complete within a single sprint is called the Team Velocity.  You need to have a gut feel for how much your team can do in your first few sprints, until you get enough history of doing Agile development to understand the velocity of your teams.  Once you have this history, it is easy to assign work to sprints up to three of four sprints in the future, because you have an understanding of how your teams perform.

There is a pile of articles and blogs on Sprint Planning.  What we have seen work the best is when teams commit to doing about 60% of their capacity in a sprint.  This leaves them time for unanticipated surprises, bugs, and other things. It also allows them to build confidence and meet their commitments.

If a team notices that they are half way through their sprint, and that they are almost done, they can then go back to the backlog (or even a future sprint), and pick one or more appropriately sized stories to complete early, in the remaining time that they have in this sprint.  Nobody has ever complained about getting functionality early.

Sprint planning is only half done at this point.  You now have a set of user stories that you are committing to implement in your sprint.  But HOW are you going to do it?  You’re going to have the entire team take those user stories and break them down into the discrete tasks needed to implement those stories.  Look at your sprint backlog now, and on the top story click on the icon for “Open child task breakdown”.  You’ll see that label when you hover over the icon.

SprintPlanning
Sprint planning – creating the tasks for your user story

Once you click on the icon, a new screen appears.  This is your area to enter all of the child tasks for the story that you are breaking down.  You should just click on the plus sign to create a new task work item, that will be a child of the user story that you selected.

TaskEntry_1
Starting your work item entry – use icon to designate this as a task

As you enter in some text to define your new task, you’ll also notice some icons below the text area.  Use the icon on the far left to set the work item type.  In our case, we will want to make this a task.  Notice that this adds some additional “shorthand” to your task description.  Try the other icons, and become familiar with the attributes that you can change in your work item.

At this point you should be getting used to how the work items work within a sprint.  You could easily decompose a large story into smaller stories, making them children of the larger story.  Don’t go crazy with parent-child relationships, use them where it makes sense.  It can be effective to have everyone looking at the same screen during these planning meetings, so the whole team is aware of the tradeoffs discussed, and the whole team has input on the estimates and work breakdown.

Use the icons to set attributes for your work item
Use the icons to set attributes for your work item

Work with your team to define all of the tasks needed for the completion of the user story.  When you are done, you should have a list of tasks that are in the Open column.  Now click on the small down arrows at the bottom of the first task in the list, and you will see some detail information for your task.  You and your team should now look at estimating how much effort each task will take.  You can also further clarify things if you need to.

Adding estimates to your tasks
Adding estimates to your tasks

Go through all of the tasks for this story, and then do the same exercise with your team for the remainder of the stories in your sprint.

At this point you are almost done with your Sprint planning.  You now need to go back and review what you have done, and check your estimated work against the calendar.  Navigate back to your original sprint planning page, and then click on the link for Team Progress.

Checking your sprint estimates
Checking your sprint estimates

You will see that you have an total of the estimates of all of your tasks in hours displayed, a number of work items assigned to the sprint, and a total of the story points.  These are all represented as the second number (0/n) in those progress indicators.  Make sure that you have not overcommitted to the sprint in terms of hours (a two week sprint, means 80 hours of work for each person on the team), or in terms of story points.  Also make sure that you have not undercommitted, and if you have, look for additional stories off of the backlog that you can assign to this sprint.  You’ll spend the remainder of your time in sprint planning making adjustments.

In a lot of the literature on Sprint Planning, they discuss a Sprint Backlog (which is what you are building here), and a Product Backlog (which we just call a Backlog here).  The terminology can be confusing, but in the end the important thing is that you commit to right amount of work for your sprint, and that you correctly define that work for your team.

When do we start doing work?

All of that effort, and we have not written a single line of code.  Ugh!  It can be tough sometimes for people eager to dive in and start prototyping and coding.  Planning is important – it helps us focus on the things that will bring value to our stakeholders and to the business.  There is nothing worse than working long and hard on something that never gets used, or has no value.

The daily scrum

So now that everything is planned, we can kick off our first sprint.  Most Agile teams will have a daily standup meeting.  The daily standup (or scrum) is a chance for each member of the team to briefly talk about what they are doing, what they plan to do, and any impediments or obstacles to their achieving these goals.  These are sometimes referred to as The Three Questions.  It isn’t a status meeting.  It’s a chance to share technical news, and other team members are encouraged to share information that can help other members of the team.  The daily standup can be run in a number of different ways, and there are a lot of mistakes that you can make in how you conduct these meetings.

Most people suggest running these daily standup meetings in front of the scrum taskboard.  You can do this with IDS by using the Team work view.  Just go to the original sprint planning view, and look at the nav bar on the left.  In the nav bar, there is an entry for “Team’s Work”, click on it and you will see the team view of the work in your sprint.

Scrum task board - with viewing options highlighted
Scrum task board – with viewing options highlighted

By choosing the viewing option icons at the top right of the view, you can look at your work as a list, as a table, or in lanes.  Most mature scrum teams will choose to view things in lanes.  If you look at your view, you will notice a lane for open work items (which have not started), and In Progress work items (which are currently being worked on).  If you look closely, there is a horizontal slider at the bottom of this view, and you can scroll to the right and see a lane for completed items as well.  Make minor updates to the taskboard as you go, as long as it doesn’t impact the meeting.  You can drag and drop work items between the lanes, and you can update the attributes if needed.

During the standup meeting, members should be notifying others of their progress, what they plan to do next, and highlight any risks or impediments to their progress.    This not only makes every team member accountable to the team (who is going to stand up and say, “Today I am going to finalize my fantasy football lineup, and watch YouTube videos of kittens?“), it also helps team members to collaborate.  They can help each other with suggestions, short cuts, and help avoid known technical issues.  You want to foster a collaborative team environment, not a structured status reporting environment (filled with Scrum zombies).

Ending the sprint

As you get closer to the end of the sprint, most of your work items should be in the resolved column in your scrum board.  You should be seeing steady progress throughout the sprint.  Your sprint should end with a demonstration of running software that demonstrates the new functionality that has been implemented.  Once the stakeholders sign off on the implementation, your sprint is almost done.

At this point you should bring your team together and have a sprint retrospective.  Go out and create one more work item for your sprint (you should do this at the beginning of the sprint, so people don’t forget it at the end).  They type for the new work item should be “Retrospective”.  During the retrospective, the team should reflect on what they did well, what they struggled with, and what they can improve.  All of this information should be captured in the notes section of the Retrospective work item.  Once the retrospective is over, the Scrummaster may want to create some new work items to reflect things that the team can do in future sprints, that will improve overall team performance.

Often these retrospective observations will end up creating stories that do not serve an end user need, but instead that serve a development team need.  Some people have referred to this as technical debt, but there is some argument about how to best characterize this work.  Examples of stories coming from retrospectives may include the refactoring of portions of the code, development of automated testing scripts, implementation of build and delivery automation capabilities, and other related issues.  Often these “tech debt” stories are the ones that are pulled from the backlog when a team finishes their planned sprint work early, and is able to take on additional work within a sprint.

Wrapping Up

At this point we’ve done a quick overview of how scrum teams can use IDS to help organize and track their sprint planning and execution, how they can groom their backlogs, and do planning in a transparent manner so all stakeholders and team members can have a shared understanding of the goals and the progress of the scrum team during a sprint.

In our next article, we’ll touch on some more intermediate topics.  We’ll talk about how to effectively use dashboards with IDS, effective ways to work with code in IDS, and some small tips and tricks that can help your teams use IDS to deliver end user results that will make your customers smile.