Homepage ->  Individual Weblogs -> Alberto Savoia  -> eXtreme Feedback for Software Development

April 01, 2004 - eXtreme Feedback for Software Development

“The road of excess leads to the palace of wisdom ... for we never know what is enough until we know what is more than enough.” — William Blake

One of the main themes of eXtreme Programming (XP) is that if an activity is beneficial and worth doing, it should be done to the extreme. If testing is a good thing, for example, we should be testing all the time; in XP this translates to customers doing functional testing and developers writing lots of unit tests and running them several times a day - even several times in a 10-minute period in the most extreme cases.

In this article, I share my initial ideas and experiences with the concept of eXtreme Feedback (XF) and its application to some key activities and metrics associated with software development. Although the inspiration for eXtreme Feedback comes from the basic ideas and themes of XP, most of them are applicable to software organizations that don’t practice XP.

Overview of eXtreme Feedback

I always knew that a sensible use of metrics and objectives was essential for making a software development team more focused and effective. However, I did not think much about the actual feedback mechanisms for those metrics until one of my colleagues, Jeff Fredrick, brought my attention to it.

As a pre-XF development manager, I typically met with the team to discuss our high-level mission (for example, release version 2.3 by June 30), get them to agree on specific objectives (such as, No priority 1 or 2 bugs and a 20% performance improvement) and then occasionally check our progress against those objectives. I assumed that everyone was on the same page, and I was right … at least for the 15 minutes immediately following the meeting.

But the focus established at those meeting quickly dissolved as various demands and unrelated activities started to interfere with it: “Mark, can you please submit a paper for this conference?”; “Mary, can you spend a couple of days reviewing this design for next year’s release?” Most of the time we met our key objectives on time anyway, but sometimes we didn’t. And even if we did meet them on time, it was with a lot of last-minute panic and heroic sacrifices that would not have been necessary had we kept our objectives clearly in sight and knew exactly where we were at any given time. Sound familiar?

Jeff, on the other hand, was a big fan of feedback. If we set a goal, for example, zero P1 and P2 bugs, he went directly to a centrally located whiteboard, drew a big matrix, and updated the bug counts several times a day. Very quickly other developers started to update the board themselves. The board got their attention. When the count of P1 and P2 bugs went to zero, people cheered; when bugs crawled back they boo-ed. It was both effective and fun. However, I am not the kind of person that can leave well-enough alone. I decided to take Jeff ’s good example and understanding of the importance of feedback one step further: eXtreme Feedback (XF) was born, named as an homage to eXtreme Programming, which, of course, includes feedback as one of its core values.

The next few sections cover some of the basic principles, tools, and techniques of XF. Use them alone or in combination, and have fun inventing some of your own.

eXtreme Focus

Even though feedback is a good thing, too much feedback on too many fronts can quickly become too much of a good thing. Generally speaking, I don’t recommend that any team use XF on more than two or three key metrics or activities at once.

One of the most valuable benefits of properly applied XF is to bring into focus what is most important at any given time. During the lifecycle of a software project, activities and priorities of a software project change, so you should change the XF targets to keep the focus on the right targets.

In the early stages of development, for example, your focus might be on measuring progress in implementing functionality and making sure that the development of unit tests keeps up with the development of the rest of the code. So during this stage, your XF metrics might be:

As the project approaches the delivery date, and most of the functionality is complete and unit testing is where you want it, the XF focus might switch to track the following metrics:

By following the principle of extreme focus, you will ensure that everyone in the organization knows what the top priorities are at any given time. This is extremely important, but it rarely happens automatically and should not be taken for granted.

Shifting the focus of XF from one phase of the project to the next does not mean that you stop tracking the previous metrics. Those results should continue to be reported, but in a way that makes it clear that they are not the current focus for the team. You accomplish that by taking them off the extreme visibility list, as the next section explains.

eXtreme Visibility (or, Fun with Feedback)

There are many ways you can deliver feedback on the selected activities and metrics to team members and interested observers:

Unfortunately, these modes of communication are not very compelling. We get dozens of emails every day, web pages require us to actively go and seek the information, information on white boards is easily ignored, and meeting announcements … well you see where I am going.

We live in an over-communicated society, and every bit of information has to compete for our attention. For XF to be most effective, we need to display information in ways that make it stand out. This is not an easy task, but fortunately solving it can be a lot fun because it will engage your team’s creativity.

Below are some ideas for XF displays you might try. Some are more extreme than others, and a few are really off the wall. Guess which ones are likely to be most effective.

The Dedicated XF Monitor

I’ll admit that this is not very extreme, but it’s already much more than most people do and it does look somewhat cool. Just hook up an LCD panel (or a plasma screen if your budget allows) to a computer that regularly monitors and updates your target XF metrics and displays the results visibly and unambiguously.

Below is a picture of one of my first XF experiments—which is used to this day. Having clean nightly or hourly builds (that is, builds that pass all the tests) is one of the things we really care about because we don’t want to accumulate a bunch of bugs that have to be fixed all at once. So I wrote a Java program that polls the results of the most recent build for some of our key projects and displays the name of the project in red if the build failed and green otherwise. A little later I added another page to display the number of open bugs—a particularly interesting metric as we approach release time.

photo 1

Photo 1: An XF Monitor having the desired effect

The display is posted in a very visible area of our building and everyone in the company (including our CEO, VP of Marketing, and even the sales team) knows what it reports. Since I sit near the monitor I hear people making comments about the metrics several times a day, and when a particularly challenging or important metric changes (from good to bad or otherwise) everybody notices: “We finally cleaned up all our P1 bugs!”; “Someone broke the nightly build!”

The cost for implementing this XF solution is minimal since you probably have an unused PC that will be more than adequate for running the monitoring application. I spent about $500 for an LCD monitor and a wall-mounting bracket. The software (written in Java) polls our build machine and our bug database (Bugzilla) and updates the results every few minutes. It took me only a couple of hours to write the software: ten minutes for the guts of the code, and almost two hours to tweak the fonts and graphics layout to make it look good.

The Ambient Orb

The Ambient Orb (see picture below) is a device that glows in different colors to report the state of whatever you want to monitor. Out of the box the Orb is programmed to track the stock market and glows green if the market is up and red if it’s down; if there is a major market move to the upside or downside, the color is complemented by a flashing action to get your attention. The information it reports can be easily customized (for example, glow green for no bugs, orange for low-priority bugs, red for outstanding P2 bugs, and flashing red for P1 bugs.)

photo 2

Photo 2: An ambient ORB reporting that all is well

What makes the Orb cool is that it works over a wireless national network and all you have to do is plug it in an AC outlet. If you really want to keep track of what your team is doing all the time, you can even take the Orb with you on vacation, put it on your hotel room’s night stand.

Ambient Orbs are made by Ambient Devices (www.ambientdevices.com) and I bought mine from ThinkGeek (www.thinkgeek.com). Unfortunately they cost $150 and you have to pay a subscription fee to use the wireless network for customization. As much as I like the Orb, I don’t think I will be buying many of them for our own XF.

Java Lava Lamp

It’s OK to break a build once in a while; it’s actually a good sign that the tests are catching problems. If it’s important to get it fixed as soon as possible, a lava lamp makes a great XF device for the purpose. Since it takes a few minutes for the lava in the lamp to start bubbling (time enough to fix minor problems - which should be fixed ASAP) the feedback mechanism matches the desired behavior: fix the problem before the lamp starts bubbling.

photo 3

Photo 3: Lava Bubbles = Build Troubles

I hooked up the lava lamp in the picture to a wireless home automation system called Firecracker (made by X10: www.x10.com) and wrote a small Java program that checks the build status through our intranet and then interfaces with the X10 through a wireless device to turn the lamp on and off. Very cool—if I may say so myself.

eXtreme Location Postings

Posting metrics on piece of paper or whiteboard is irresistibly cheap and very easy, but it’s not very extreme or very effective - unless you apply some creativity in picking the location. Your company’s bulletin board is not the right place.

Below is an XF posting that will probably get noticed by virtue of its location. Make sure you enlist a member of the appropriate gender to help you if you want to cover both the men and women’s bathrooms.

photo 4

Photo 4: No comment

The Possibilities Are Endless

Use the examples I have given you as a starting point. I am sure you can come up with more innovative and effective way to deliver XF, and don’t limit yourself to visual feedback, the field is wide open for pioneering new XF devices based on sound or smell.

eXtreme Openness

Extreme openness means that the feedback on the activities and metrics that you have selected as the current focus of your XF should be very visible and shared broadly; not just within the development team, but throughout the organization. You might even want to share some XF data with your customers - something we are planning to do at my company.

Let’s assume, for example, that your development project is nearing completion and your release readiness criteria include the following:

You clearly want this information shared with the entire team. If your project is critical to the company, there is no good reason why this information should not also be shared with the rest of the organization. The VP of sales might be depending on the release to make the numbers for next quarter, the customer support organization needs to know when they have to be ready to answer questions about the new product, etc. And if you have some key customers that are waiting for this release, I am sure they’d like to know where you are.

Thanks to the Internet, achieving extreme openness with your internal and external customers is as easy as setting up a URL with the key information. Use a password if you want to limit access.

eXtreme Frequency

The update frequency for an XF metric should match as closely as possible the rate of change of the target you are measuring. If you are monitoring the status of a nightly build then a daily update is enough. But if you practice continuous integration it makes sense to update the associated XF display along with the build results. Ideally the XF display should be updated in real-time since even small delays will cause people to complain: “Hey! I fixed the build 20 minutes ago; why is the lava lamp still on?”

eXtreme Rewards

XF is much more effective if there are rewards associated with achieving the targets on which your team is focused. In the XF tradition and for maximum effectiveness, you should try to make the rewards as memorable and unique as the display. Just taking the team out to lunch, for example, is nice but not really extreme or memorable. On the other hand, if you take the team out to lunch to a place like the Prince of Wales Pub in San Mateo, CA, where they serve the infamous and impossibly hot Habanero Hamburger, I can guarantee you that the extreme laughter and tears that will accompany the event will make it memorable.

photo 1

Photo 5: To the survivor a bumper sticker

I am not going to dwell on this aspect of XF because there are plenty of books and resources on creative and extreme ways to reward employees and teams; check them out. Even better, ask your own team what they consider to be memorable and extreme rewards.

eXtreme Results

My personal experience with eXtreme Feedback has been eXtremely Positive. Objectives that would normally be ignored, or be given lower priority than they deserve get done quickly and effectively after they become the focus of XF.

If anything, at times XP can be too powerful a tool, especially if people abandon common sense in the pursuit of the XF objective: “I did not return that customer support call because I was busy getting the bug count down to zero.” So, if that’s the case, make sure that the team knows that there are overarching objectives that should take precedence over the XF objectives; for example, “Returning customer support calls should always be your top priority.”

Conclusion

Your software organization will be much more effective if people use a commonly agreed set of metrics and work in unison toward achieving a small number of high-priority objectives.

The techniques and tools of eXtreme Feedback help you to achieve and maintain focus on what is most important to the organization at any given time. By making the feedback mechanism fun you help to draw attention to it. By making the feedback broadly available you send out a signal that the objectives are important and that many people care about, and depend upon, their achievement.

Extreme feedback worked for me, and I am confident that you will achieve similar results; but there’s only one way to find that out for sure …


Posted by Alberto Savoia at April 1, 2004 07:48 AM


Trackback Pings

TrackBack URL for this entry:
http://www.developertesting.com/mt/mt-tb.cgi/117


Comments

Post a comment




Remember Me?