August 11, 2006 - Old metrics never die
If a little bit of feedback is good, a lot would be even better, right?
Is there such a thing as too much feedback?
XP doctrine says that you should stop tracking metrics once they have served their purpose. You should only have 3 or four "Things To Focus On". There's a reason for that.
One under-appreciated side-effect of hand-drawn metrics or charts or reports is that, once they stop providing value, they die off through neglect. Automatically generated reports don't have that property.
Once you automate a report it lives forever. You are unlikely to accumulate more than about three or four hand-drawn metrics because they take so long to prepare but if you automate them...
It takes longer to get going with an automated metric but once you have it working it will happily run forever. You can easily get to the point where you are swamped with hundreds of emails telling you that the build succeeded and that the every growing battery of tests is passing. If you are like us, eventually you'll start to ignore the messages.
If you have thousands of tests running, there will come a time when tests will start failing (if this doesn't happen, you should wonder whether the tests are any good).
Usually you will want to address those failing tests right away but it won't always be convenient to do that. Worse, if you write acceptance tests for future stories they are going to continue failing until the feature gets implemented.
To add more misery to our story - failing tests enjoy company. If there is one lonely, little failing unit test you can GUARANTEE that he won't be lonely for long. Before you know it, he'll have invited a whole bunch of his failing friends over. It's also a scientific fact that Red Lava Lamps attract failing tests like flames attract moths. If the red light is on, all the failing tests for miles around will gather around it.
If a test is failing, fix it or, if that is not possible right now, make sure it does not cause the build to fail. There are bunch of tricks to achieve this and I'll cover some of my favorites in another post.
If a metric is no longer relevant, eliminate it or the good ones will get lost in the noise. Do you really need all those reports?
One trick I use to deal with CruiseControl spam is to set up two filters. The first moves everything from cruisecontrol@agitar.com to a separate folder. The second marks messages as read unless they have the word "failed" in the subject. I don't even need to see the endless stream of success messages unless I am waiting for confirmation that my check-in succeeded.
Every piece of data that is not actionable is a distraction. Get rid of it. Be brutal.
Posted by Kevin Lawrence at August 11, 2006 01:34 PM
Trackback Pings
TrackBack URL for this entry:
Comments
Why not mail yourself only on failures?
Posted by: Jason Yip on August 11, 2006 04:57 PM
I like to get the confirmation that my checkin was successful. Sending it to separate folder is a compromise. I get subliminal comfort when the conformation email arrives but minimal distraction because I don't need to read it.
Posted by:
Kevin Lawrence ![[TypeKey Profile Page]](../../nav-commenters.gif)
on August 15, 2006 07:35 AM
Post a comment
W3C link checker Powered by Movable Type 3.16
This weblog is licensed under a Creative Commons License.
Agitar, Agitator, Software Agitation, Open Quality, and Quality Level Agreement are trademarks of Agitar Software, Inc.