On this blog when we talk about developer testing we have an ongoing assumption that you're going to use continuous integration in both senses of the phrase: you're going to check-in frequently and you're going to have an automated system like CruiseControl running your tests.

But, perhaps surprisingly, many teams have as much trouble getting the automated system up and running as they do in writing the tests in the first place. The problem is their build. They have a system that is sufficient for assembling their software but that isn't ready for the demands of an agile team, demands like continuous integration, unit tests, functional testing, software metrics, etc.

If you have this problem then you should take a look at the Build Engineer Bootcamp that we have scheduled for Minneapolis in April to coordinate with CITCON, the Continuous Integration and Testing Conference. These two events should pair very nicely: attend the bootcamp to learn the basics, then attend CITCON to delve into the state of the art.

Hope to see you there!

I've written up my account of how I spent my time at Elisabeth Hendrickson's Open Source Test Automation Tool Love In. My goal was to have a single script using Watir to drive both Safari and Firefox. With a bit more work I'd like to have the same script driving Celerity. I like the idea of a continuous integration setup where the Celerity tests run first for fast feedback and then the tests run in the browser to ensure compatibility.

2.8 is now available for download.

Along with a number of bug fixes and enhancements (release notes) there have been a number of significant below the surface changes that should be invisible out of the box but will be a GoodThing:

• Java 5 required to run: this allows us to update the code to more modern Java and to take advantage of the libraries that are part of Java 5 but not Java 1.5. (Of course you can still compile your projects with JDK 1.4.)
• Upgraded Jetty: along with the upgrade we are also exposing the Jetty configuration files to allow more extensions to the default behavior.
• Updated to Servlet API 2.5 and JSP API 2.1
• Specify Log4j config file: there is now a command-line switch (-log4jconfig) to specify the Log4j configuration file. Importantly, this can be either in the .properties format or the .xml format. The .xml format is more verbose (no surprise) but also allows more configuration options. From the Log4j tutorial:

"The properties file does not support some advanced configuration options like Filters, custom ErrorHandlers and a special type of appenders, i.e. AsyncAppender. ErrorHandlers defines how errors in log4j itself are handled, for example badly configured appenders. Filters are more interesting. From the available filters, I think that the level range filter is really missing for property files."

In addition to these change there quite a few little hygiene cleanups that made me happy.

CITCON Amsterdam is October 3rd & 4th and the final day to register is tomorrow, September 19th. We currently have 124 people registered from across Europe, ready to talk all about Continuous Integration, Testing and related topics. Hope to see you there!

In this blog entry I talk about how simple changes can produce dramatically different results. Developer testing is one of the practices, one of those little things, that add up to excellence:

"… there is no secret; there is only the doing of all those little things, each one done correctly, time and again, until excellence in every detail becomes a firmly ingrained habit, an ordinary part of one’s everyday life."

Read more.

CruiseControl 2.7.2 is available for download.

Lots of bug fixes, lots of changes to the Dashboard.

New plugins:

• Cincom Smalltalk VisualWorks Store Source Control
• Git Source Control & Bootstrapper
• Http Publisher
• Mercurial Source Control & Bootstrapper
• SVNLabelIncrementer

See detailed release notes here.

The other software metric to come out of Agitar, CRAP is simply too much fun. So David Kao (a former Agitar intern) and I decided to take the levity down a notch.* The result?

CITCON Denver is just around the corner, April 4th and 5th. This the 3rd annual CITCON North America event, the previous years having been in Chicago (2006) and Dallas (2007). CITCON events have also been held in London (2006), Sydney (2007) and Brussels (2007).

For those who haven't heard about it CITCON is the Continuous Integration and Testing CONference, an open spaces (or "unconference") event that attracts an energetic and enthusiastic crowd of CIT practitioners from around the US, Europe and Asia. If you've never been to an open spaces event you can think of it as the best part of a normal conference -- the hallway conversation -- dialed up to 11!

You can read about past CITCONs on the wiki and then register today. But hurry, there are already 120 people registered and only 30 more registrations will be accepted!

If you can't make it to Denver then you might want to plan ahead by registering now for CITCON Melbourne in June or CITCON Amsterdam in October. Or maybe even all three!

Artur Hildebrandt is a MBA student at the University of Liverpool and currently running a survey on the business benefits and risks of unit testing. His goal is to identify and measure benefits software development organizations can gain by actively practicing unit testing. I think it would be great to have more information available to persuade management on the benefits of unit testing so I'm happy to promote his survey. If you'd like to take part his questionnaire available at online.

I'm sitting here in SFO waiting for my flight to Tokyo. I'll be speaking this week at JaSST, the Japanese Symposium on Software Testing. This will be a bit different from my normal talks on CI because I'll be focusing on the benefits of CI for testing groups rather than my normal focus on developers. And there will be translation into Japanese which makes for a very different presentation experience. But I'm very excited about giving this talk because Continuous Integration is less well known in Japan than in Europe, Australia or the US, so it will be fun to be bringing the message to a new audience.

Several prospects and friends in the software development business have asked us to make a downloadable eval for AgitarOne available on our web site. It is finally here!

It would be handy to have a report of all the unexpected exceptions that get thrown.

I've lost count of the number of times I've heard that. I've even said it myself a few times.

It seemed like it would be pretty easy to do using the dashboard.xml feature of AgitarOne, so I decided to give it a try.

Crap4j 1.1.6 is out. The new version features historical trends of CRAP metrics, and comparison by similarly tagged projects. Exciting!

First we generate over a million free JUnit tests and then we get nominated for a Jolt Award.

What a great end to the first year of JUnit Factory!

Configuring CruiseControl the CruiseControl way from ThoughtWorks Studios presents what should be a standard practice. Checking your cruisecontrol configurations into version control and having cruisecontrol bootstrap with them.

Now I could see it getting tricky if you have multiple build machines, but it seems like that would be pretty easy to handle as well by using different config files for different machines, or something similar. That might violate the DRY principle, so it would be good to think a little more about it. How do others deal with this?

Like the title says.

This is particularly exciting because now it can be included in continuous integration setups.

One natural next step would be to set a threshold value so that crap4j would throw an error that would fail the build. I would be curious to hear comments on how people would like that to work.

Anyway, find out how to get the latest version at the blog.

At CITCON Europe in Brussels last week one of the sessions I enjoyed was on CRAP4J and other metrics for bad code. (I've put my notes up on the CITCON wiki.) Today Kevin reminded me that Clover has a similar metric for identifying risky code, a tag cloud that uses complexity to size the tag and the coverage level to color it. They have posted a sample using Lucene here. This is a pretty neat looking approach... but honestly? I don't really like it.

A new version of crap4j has been uploaded to it's new home at http://www.crap4j.org/!

Please check it out and give us your feedback!

CITCON Europe 2007 starts today in Brussels, Belgium. We're in a nice central location, very close the Brussels North train station. As an open space event everyone has the opportunity to propose a topic for discussion. The one I'm most interested in is talking about Crap4J and other ideas for metrics to spot bad code.

It is a curious fact that, if you say

Let them eat cake!

at an international gathering, the French-speaking people will have no idea what you are talking about. Even if you say it in French.

A History of Build Systems

In my younger days, before I knew any better, many projects I worked on compiled and published their software manually. You'd type cc and then copy these bits over there and then zip that directory and post it to there. Eventually, we figured out we could write little scripts to automate all the tedious bits and make it less fragile and more repeatable.

One day, I discovered the discipline of daily builds and tools like make and my life got a whole lot better. Make gave us, in Elizabeth's handy phrase "a place to put things".

Elizabeth Hendrickson is a tremendous facilitator and a canny manipulator.

In Influence: The Psychology of Persuasion, Robert Cialdini describes various techniques for making people do things that, if they were thinking clearly, they would otherwise not do because of lethargy, laziness, or because it would offend their better judgment.

One of those techniques is The Commitment Principle which was used on American POWs to great effect by the Chinese during the Korean War.

My normal response to a NoClassDefFoundError is to ask google. This generally gives me a thread I can follow and eventually solve the problem, but it isn't very efficient. It looks like a better solution would be the What Jar? website... if only there were more jars in the index.

Have some extra time on your hands? How about uploading a bunch of the jars to save me some time in the future? :)

Kevin says all the cool kids are doing it...

Alberto blogged today about our free tool Crap4J. You might think that the last thing software development needs is another metric, but our goal here was a bit different. We were looking for a metric that would be simple and actionable like the cholesterol index: if you know your cholesterol score is over 200 you know you need to do something, it is a call to action. By the same token, you can't say that just because your score is under 200 that you're healthy. The cholesterol isn't a perfect indicator of health, and yet it is still useful.

Did we succeed in our goal? Check it out and let us know...

(And thanks to the writeup on The Server Side for my title.)

Kevin pushed a new version of JUnit Factory yesterday. The coolest new feature is that you can now execute your hand-written tests remotely on our server and get a coverage report out of it, such as this dashboard report for CruiseControl.

To make this work safely for us we run the tests under a security manager and restrict what the tests can do, so some tests will fail. But if your tests are clean according to Michael Feathers' Set of Unit Testing Rules you'll be just fine.

"Wow!

I have just generated unit tests for some code that would have taken months or years to do manually and I did it in under 30 minutes including registering on your server and waiting for the reply email.

That is truly awesome!"

Getting this kind of feedback is the fun part about having free (as in beer) software up on the web where anyone can try it out. In this case Nick had a great experience with JUnitFactory and let us know with a post to our forum.

As Kevin posted on the Agitar Developer News page, three of us will be at Agile 2007 this week. Bob is offering his tutorial "To Catch A Bug, Think Like a Bug" and I'll be leading a discovery session with Alistair Cockburn on "Creating Change One 'Tic-Tac' at a Time". The rest of the time you'll find me at the Agitar booth, so if you'd like to talk unit testing, continuous integration, or just hang out, stop on by and say hi.

Julio doesn't like my blog -- it's too impersonal, not enough of me here for a friend. He's right of course, but I think Rands might have given me a solution in Twitter.

The problem is the tyranny of the pending entry. I typically have a queue of dev/testing/ci topics I want to blog and I find it hard to let more personal items jump the queue. In theory I could ignore that I still haven't shared the advice in On Writing Well that sounds like refactoring and dash off a note that Leanne has started he own softgoods workroom... but only in theory. My mind just doesn't seem to work like that.

Today though I read Rands description of Twitter as a yard sale for casual information and was interested. Then I read "I'd prefer to be drinking with y'all, but I'll take what I can get" and knew I had to sign-up.

If you're thinking it's been too long since we've had a beer together maybe you should sign up too?

Tonight I'll be presenting a modified version of my upcoming Agile 2007 talk, To Catch A Bug, You Have to Think Like a Bug at the BayXP meeting hosted at Google in Mountain View.

I might also show off the new crap4j tool that we have been working on lately.

"…AgitarOne is an easy-to-use workgroup product that greatly facilitates the use of unit tests and helps sites get as much benefit as possible from this activity. The result is shorter QA and debugging cycles and much better predictability of the software process. For many sites with large, important Java projects, this solution is attractive and compelling. All such sites are likely to derive value that far exceeds the cost."

I'm very excited that with two weeks remaining before the conference we've reached the 100 mark for registrations for CITCON Sydney. This is our fourth CITCON event but our first in Australia, so it is great to see so many people registered. We'll take up to 150 so if you're interested in attending, it's not too late to register. The conference will be July 27th and 28th, Friday night and then all day Saturday.

Btw, if you're not familiar with the Open Space conference format you might want to read this description. If you're still not sure if this format or event is for you, you might want to see what people have posted about past CITCONs on the web, or read the feedback from Dallas earlier this year.

We're always looking for bright engineers to add to our already amazing engineering team - but we're specifically looking to fill two positions ASAP - Senior Software Engineers for our core engine development and quality engineering teams.

Those of you familiar with our product would know that we solve some very tough problems ... most of our products work on problems which are defined NP-hard and then solve them to a reasonable and usable level.

So, if you're looking for a challenge and want to be surrounded by equally smart and bright individuals, if you dream algorithms, and if you have a passion to write and test software, then read the posting and apply for either the Senior Software Engineer or the Senior Quality Lead Engineer; other positions are also listed at http://www.agitar.com/company/careers.

If you know someone else that could be a better fit, I would appreciate your forwarding this post to them ... thanks for your help!

The recent 2.7 release of CruiseControl sees CC join the modern age with a completely new web application called the dashboard. The dashboard provides an attractive web 2.0/ajax interface, a widget based extension system, and a bunch of new features like updating your CC configuration via the web. The old reporting application is still there -- meaning all the 3rd party tools still work -- so there is no reason not to update and check out the new features. Enjoy!

I like how Alistair is bluring the lines between Wiki and Blog on his site. Martin does something similar, but his Bliki is read only. Alistar, otoh, is using MediaWiki to run his full site, including his blog, and encouraging people to post their thoughts on the discussion page for the related blog.

I like the informal collegial feel and I'll be interested to see if a community does in fact develop.

Reading Paul Duval's blog entry I found him making a comparison I use frequently: "CI provides you with a mirror of your software under development." His point is that if you find yourself getting mad at the build you might want to check yourself.

If you don't like what you see in the mirror, do you blame the mirror or do you start thinking about spending more time at the gym?

Tomorrow night, I will be giving a talk at the Association of C and C++ Users in Silicon Valley entitled, "To Catch a Bug, You have to Think Like a Bug". This is a new and improved version of a talk I gave at SD West, so if you didn't get to go there, you get another opportunity to check it out. I hope to see some Agitator's there. You can find out more of the details at the ACCU's website.

This picture was just about the last quiet moment before they opened the exhibition hall doors. Shortly after that the horde decended and at the booth we've been just shockingly busy.

I'm encouraged by the number of people who are interested in unit testing and want to hear our pitch. We've been handing out JUnitFactory business cards like candy -- the ability to generate dashboards is a killer new feature -- and the printed copies of The Way of Testivus are popular as well. (I'm still waiting to see someone put on our karma > dogma t-shirts. Maybe tomorrow..)

I've been mostly busy occupied describing AgitarOne to people but in between I've managed to catch up a bit with Paul Hammant (who is pleased I finally have the include.projects working so he can Branch by Abstraction) and Andy Glover (who is threatening to join us in Belgium). Andy introduced me to prolific traveling speaker Neal Ford, who must enjoy being on the road far more than I do. I got within handshaking distance of ASM author humble committer Eugene Kuleshov but was too busy talking with someone in the booth to do more than exchange greetings.

Looking foward to my next break and the opportunity for some longer conversations...

I am going to be attending sessions at JavaOne this week, and would be happy to meet with any Agitators or testing enthusiasts at the conference, according to Sun's Event Connect tool, I can paste this code and you can link to me in their event tool to set up a meeting.

I also set up a topic proposal for the JavaCamp, unconference that is happening Tuesday and Wednesday nights, on Adding JUnit to the Java Platform.

Here's the blurb for anyone interested:

Many language platforms, like Ruby and Microsoft .NET ship with a unit testing framework as part of the platform. Why not include JUnit in the Java Platform, or at least include it in the JDK? Code quality is a constant sore spot for commercial applications, so it seems like making the tools that contribute to higher quality more widely available will encourage better code. While we're at it, lets put in a code coverage tool as well, so we can see how well we're testing. We already have some profiling and management tools built in, so this seems like a missing piece of the puzzle.

I'd like to discuss this idea, and concerns around improving code quality with developer testing in general.

JUnit Factory has a new feature - project dashboards - and I thought I'd try it out on my bowling code.

Reading John's post reminded me that I should let people know I'll be at JavaOne this week on booth duty for Agitar. My favorite parts of these events are all the side conversations so if you want to talk about CruiseControl, CITCON, developer-testers/tester-developers, OIF, how to run an open spaces conference, JUF, adopting developer testing, introducing change in general (I'm looking for material for my 'Tic-Tac' session with Alistair Cockburn at Agile 2007), or The Way of Testivus (PDF) stop by the booth or send me an email and we can set up some time to talk. (Following Bret's session I'm really interested in hearing from people who have had good experiences using RSpec on a Java project.) See you there...

Referring to "The Way of Testivus" entry:

http://www.artima.com/weblogs/viewpost.jsp?thread=203994

Morgan Conrad asked: "What is Testivus' wisdom concerning the proper percentage of test coverage?"

Here you go Morgan:

Testivus On Test Coverage

Early one morning, a programmer asked the great master:

“I am ready to write some unit tests. What code coverage should I aim for?”

The great master replied:

“Don’t worry about coverage, just write some good tests.”

The programmer smiled, bowed, and left.

...

Later that day, a second programmer asked the same question.

The great master pointed at a pot of boiling water and said:

“How many grains of rice should put in that pot?”

The programmer, looking puzzled, replied:

“How can I possibly tell you? It depends on how many people you need to feed, how hungry they are, what other food you are serving, how much rice you have available, and so on.”

“Exactly,” said the great master.

The second programmer smiled, bowed, and left.

...

Toward the end of the day, a third programmer came and asked the same question about code coverage.

“Eighty percent and no less!” Replied the master in a stern voice, pounding his fist on the table.

The third programmer smiled, bowed, and left.

...

After this last reply, a young apprentice approached the great master:

“Great master, today I overheard you answer the same question about code coverage with three different answers. Why?”

The great master stood up from his chair:

“Come get some fresh tea with me and let’s talk about it.”

After they filled their cups with smoking hot green tea, the great master began to answer:

“The first programmer is new and just getting started with testing. Right now he has a lot of code and no tests. He has a long way to go; focusing on code coverage at this time would be depressing and quite useless. He’s better off just getting used to writing and running some tests. He can worry about coverage later.”

“The second programmer, on the other hand, is quite experience both at programming and testing. When I replied by asking her how many grains of rice I should put in a pot, I helped her realize that the amount of testing necessary depends on a number of factors, and she knows those factors better than I do – it’s her code after all. There is no single, simple, answer, and she’s smart enough to handle the truth and work with that.”

“I see,” said the young apprentice, “but if there is no single simple answer, then why did you answer the third programmer ‘Eighty percent and no less’?”

The great master laughed so hard and loud that his belly, evidence that he drank more than just green tea, flopped up and down.

“The third programmer wants only simple answers – even when there are no simple answers … and then does not follow them anyway.”

The young apprentice and the grizzled great master finished drinking their tea in contemplative silence.

Translator’s Introduction

In May 2006, an ill-prepared international expedition to the Himalayas lost its way. After two weeks of wondering around, hungry, thirsty, and smelling like inexperienced expeditioners who got lost for two weeks, they stumbled upon the entrance to an ancient cave.

Once inside, they saw a maze of ancient, and messy, cubicles. Each cubicle had a wooden desk, an ergonomically correct bamboo chair, a Dilbert™ calendar, and a strange computer-like mechanical device. In one corner of the office they found barrels of dark liquid which they later identified as early examples of carbonated and highly caffeinated drink and a ping-pong table. They realized that the cave was an ancient software start-up. The oldest one on record. Older even than Netscape.

Among the many things they discovered inside the cave was a note left by one of the programmers. The expedition’s guide, while not very good at guiding, knew how to read the ancient language and translated the note for them:

We have finished the release ahead of schedule – again. All the tests pass, so we are taking the rest of the week off. We are going sailing. Since it’s a team building exercise, we hope we can get reimbursed for it.

The explorers looked at each other in astonishment. Not only had they discovered the oldest software start-up in history, they had also discovered a team of programmers who, apparently, completed their code ahead of schedule ... on a regular basis!

What was the secret of these ancient programmers?

And what had happened to them?

The expeditioners searched each cubicle for clues and found two mysterious booklets. One of them was called "Learn To Sail In 30 Minutes”, which explained the fate of the programmers. You are holding in your hands a translation of the other booklet: “The Way of Testivus”.

Who wrote this mysterious booklet? What is Testivus? Only Google™ knows for sure.

Is the content of this text responsible for these ancient programmers being able to complete projects ahead of schedule?

We can’t be sure, but we believe that the amazing prowess of these programmers was probably due to a combination of the Testivus philosophy, and the consumption of large amounts of the dark caffeinated liquid found in the cave.

Read the booklet and draw your own conclusions.

Alberto Savoia, April 2007, Mountain View, Ca

This Friday the 13th is the final day to register for CITCON, the Continuous Integration and Testing Conference. CITCON Dallas/Ft. Worth will be held Friday April 27th and Saturday April 28th at the American Airlines Training and Conference Center which is right outside the Dallas/Ft. Worth airport in Texas.

CITCON is different from a standard conference in a couple of ways:

• It is free, but scheduled for a Friday night and Saturday. This means the people who attend are the people who are passionate enough about the topic to give up some of their personal time, rather than those people who are able to have their companies pay for it.
• It is an open spaces event. So rather than going from talk to talk and hoping to hear something of interest the attendees build the agenda, with each topic an interactive discussion.

If you've ever attended a conference and left feeling that the best part of the conference was the hallway conversation, then you should understand the appeal of the CITCON format. (At least this is the appeal for me!)

Past CITCON events (Chicago & London) had a fantastic mix of people, including CI and testing tool makers, very experienced authors and consultants, new and experienced Agile practitioners. CITCON Dallas promises more of the same.

Hope to see you there.

I came across a nice quote from Ron Jeffries in answer to the eternal question about the color of the tests that result from TDD.

I hosted a JUnit Factory presentation a few days ago (you can watch it online if you missed it first time around) and spent a fair amount of time talking about the Triangle sample in the JUnit Factory demo.

There are a handful of web technologies that I use a lot. It's handy to have a cheat sheet around for when I can't remember whether it's switch-case or choose-when or if-test-else.

Here are a few that I use all the time:

• xml, xpath & xslt
• JSP, EL, JSTL
• Regex

Any more I should know about?

Sometimes an aphorism like “good development depends on good communication” doesn’t really sink in until it hits you upside the head. I experienced the reality behind this particular maxim recently when I expanded the number of developers on an open source project from one developer, myself, to two.

I can code passably well and I am comfortable with public speaking - but there is something about combining the two that makes my brain just completely shut down.

For completeness, I run the characterization tests one last time. As you might expect, there are failures because the behavior of getSecondBall() and needsMoreBalls changed.

I am impatient to be done now, so I'll try to get through the code for spares quite quickly so that I can review my findings.

Someone asked me how the characterization tests fared after such an extensive change. After all, I added new methods, new behavior to existing methods and I refactored extensively.

With the code for strikes written, it's time to run the acceptance tests to see if they agree that we are done.

Tomorrow morning, I'll be giving a talk at SD West 2007 on developer testing. It is a a very opinionated look at how to test your code. It should be fun and useful. If any Agitators or other test afficionados are going to SD West, it would be great to see you at the talk, or afterwards as well.

Here are the details:

To Catch a Bug, You Have to Think Like a Bug

According to the rules:

2.1.3 A strike is made when a full setup of pins is knocked down with the first delivery in a frame.

After I completed the first story, I generated some characterization tests with JUnit Factory.

Before I move on to the next story, I want to revisit those tests and make sure we have not introduced any regressions.

Before I move on to spares and strikes, it would be nice to see how the score sheet looks. In an earlier post, I claimed that one of the reasons for integrating the UI early is to make sure the domain model will satisfy the requirements of the user interface. Let's see if it does.

My last post ended with this bold assertion:

If I am not mistaken, I have written enough code to pass the acceptance tests for this story.

It's at about this stage of the bowling example that people usually leap into a discussion about the appropriate data structure to store the rolls and the APIs for exposing the results.

In the previous installment, I wrote the code that implements rule 2.1.1. For rule 2.1.2, I finally start to add up some scores and show them in the score card.

Jared posted a cogent argument for peer code reviews that is worth reading if only for the killer line

With enough code, all eyes are shallow.

It's an interesting word, 'test'. It can mean so many things. Before XP came along it used to mean

find out whether something works correctly

In my previous blog entry, I posted a set of acceptance tests for the first few stories. It's time to start writing the code to pass those tests. I prefer to discover the design through TDD rather than code directly to the customer-facing tests.

I have often written acceptance tests for code that has not yet been written (in fact, I wrote an article about it) but I have never written tests that will work with any number of implementations, each with their own architecture. I don't even know how to go about it, but that never stopped me before...

"scoring a game of bowling" is probably the most common application used when demoing TDD. It's so commonly known among the JUnit crowd that I chose one of Bob Martin's efforts as a demo for JUnit Factory.

The topic comes up about once a year on the TDD mailing list and it just came up again. By an odd coincidence, we just celebrated the completion of a new release of AgitarOne with a trip to Homestead Lanes, so I am all fired up about bowling despite my dismal performance (there was beer involved).

For me today was my best day so far at an EclipseCon, but as usual for a conference (except CITCON!) the most interesting stuff was what happened outside the talks...

We are proud to announce that registration is now open for the next Continuous Integration and Testing Conference, CITCON Dallas on April 27th & 28th. Space is limited to the first 100 registrants and attendance is free. Registration is on-line and while it is open until April 13th we do expect to fill all the available slots, so sign-up soon to reserve your spot.

The conference will be following the same Open Spaces (or unconference) format as the 2006 CITCON in Chicago and London. These prior CITCON drew enthusiastic practitioners at all levels of experience, all looking to share what they knew and to learn what they could. We're looking for more of the same in Dallas.

Please help spread the word about CITCON and we look forward to seeing you there!

Paul Julius
Jeffrey Fredrick

If you're at EclipseCon this week you might be interested in stopping by the panel Making Unit Testing Part of Your Development Process: How to Get Your Team to Do It. I'll be there as a panel member... but it should be a good panel anyway. ;-)

Last week CruiseControl 2.6.1 was released. In this release there are several bug fixes but also an odd little plug-in that seeks to address a common problem with inter-project dependencies. Read on to learn about the new Veto plug-in and how it can help keep your projects building in the correct order.

I mentioned a couple weeks ago that I was going to be part of a webinar with Carey Schwaber of Forrester Research on the Business Benefits of Unit Testing. Well that webinar has now been posted so you can view the replay. You do need to register to view it but if you don't want to hear from us just put in bogus data. (So why ask for it? Because some people do want to hear from us and we're most interested in enabling those people.)

Developers need to take more responsibility for testing their code. But the majority of developers are not willing, nor ready, nor able to jump on the bandwagon of the more extreme and demanding developer testing movements such as Test Driven Development. Testivus is a proposed developer testing movement "for the rest of us".

Mocks Aren't Stubs by Martin Fowler, is a very comprehensive look at two pairs of issues in testing: state-based verification vs behavior verification, and classical TDD vs Mockist TDD.

Some developers are easily test-infected - they take to unit testing like a duck to water. Others need some time and encouragement, but eventually "get it". A third group appears to have immunity to test infection. I invent a test-gene model to categorize these groups and look at its implications for the future of developer/unit testing.

In light of the recent conversations about the adoption of developer testing on the junit list and Artima, this Turing Award lecture by Robert Floyd seems particularly appropriate. There's a particularly good quote where he is discussing a quote from Thomas Kuhn in "The Structure of Scientific Revolutions."

"Again from Kuhn:

"The older schools gradually disappear. In part their disappearance is caused by their members’ conversion to the new paradigm. But there are always some men who cling to one or another of the older views, and they are simply read out of the profession, which thereafter ignores their work."

In computing, there is no mechanism for reading such men out of the profession. I suspect they mainly become managers of software development. "

I've been so self-absorbed with my travel travails that I forgot to mention I'll be part of a webinar tomorrow morning on the Business Benefits of Unit Testing with Forrester anaylst (and seemly very nice person) Carey Schwaber. I had read some of her previous reporting on the build market, because of her mention of CruiseControl, and meeting her in person confirmed what had come across in her reports: she's pretty clueful about the the development market.

So if you're interested in the topic I think it would be worth listening. And don't worry, Carey will be the star of the show -- this time I'm just the vendor shill.

Yesterday I rashly predicted it would take me about 28-ish hours to get home from Bangalore, but now I find myself writing this from a hotel in London. It seems the relatively good luck I've had in my traveling all came to an end on this trip and over the week I've had to deal with (1) my luggage not arriving with me in Bangalore, (2) my Chennai-Bengalore flight being delayed by four hours, (3) a two-hour sit on the tarmac in Bangalore waiting for the fog to lift, leading to (4) missing my connecting flight in London. Reflecting on these minor setbacks over my bland English breakfast (I miss those idli with sambar already!) I decided there was a lesson in here on one of my favorite development/process topics, which is feedback.

It is 3:55 AM and I need to head to the airport in about 30 minutes but before my lovely 28-ish hour journey back home I wanted to jot down a repeating theme from the STeP-In Conference that just concluded here in Bangalore...

I'm in London right now, on my way to Bangalore where I'll be speaking at the STeP-IN Summit 2007 on Improving Quality Assurance through Developer Testing.

On the strong recommendation of Eddie Correia (editor of ST&P) I picked up Thomas Friedman's The World is Flat. That makes it his fault that I didn't sleep on my SFO-LHR flight! I'm not sure everyone would find is as compelling as I did, but for me it could almost be a companion book to my professional memoirs, or at least my memories.

I remember a long conversation the day Roger showed me NCSA Mosaic for the first time. It was the first time I heard the word Netscape (Flattener #2), and then we talked for hours about how the web was going to change everything! A few years later Jayson convinced me that there were some really big ideas there in Open Source (Flattener #4) and we started OpenAvenue, where we hoped to create a collaborative platform for Outsourcing (Flattener #5) and Offshoring (Flattener #6).

And as a parent who has watched the world change I've often felt The Quiet Crisis and wondered if I'm doing the right thing to prepare my children for the future. It is a question that I return to again and again. I'm living in the flat world and I know how to navigate it for myself. But as I hang up from calling my family via Skype on my way to "Asia Pacific's Largest Testing Conference" I don't know what to think about their future.

We get a lot of sample code that exhibits incorrect or unexplained behavior with AgitarOne - and many times we have to set it up in a debugger or import that code into an existing project. Recently I discovered that eclipse has a very easy technique for just this use case - thought I'll share it here.

We are working on an early version of JUnit Factory and we learned some very alarming lessons about web usability this week.

The main lesson: no one reads anything.

I'm off to Germany this week to give a few talks, some of the public. First on Tuesday I'll be at the iX Konferenz in Frankfurt giving the talk AgitarONE – Automatisiertes Java Developer Testing. Then on Thursday (Frankfurt) and Friday (Munich) I'll be part of the seminar JUnit-Test Generierung auf Knopfdruck!.

I hope nobody will be disappointed to hear that I'll be speaking entirely in American; my knowledge of German is limited to "ich bein ein Berliner".

Brian Marick is a funny guy

Those in the Agile world all know of resistance to Agile from those middle managers who see it as a threat to their power to command and control. Telling such a person that her sabotage endangers the company's ROI is like an abbot standing in the path of Christian raiders and threatening them with loss of their immortal souls: sometimes it works, but nowhere near often enough. And it never works with the worshippers of Odin.

CITCON London registration will be closing this week, it is just hard to know if we will hit the deadline of Friday September 22nd or the cap of 120 people first! At last count we have 85 people signed up and if history is a guide the remaining spots will go fast.

For those who aren't aware CITCON is the Continuous Integration and Testing Conference, a free open spaces (or unconference) event on continuous integration and the testing that goes with it.

Given the open spaces format it is impossible to predict what the exact session topics will be but a sampling of the topics (and notes) from the CITCON Chicago event from earlier this year is available on the wiki. Also available are some photos, feedback, links to related blog entries and more... More than enough to be convinced that you should sign up today!

Listened to Uncle Bob give his Clean Code talk today and took some notes. Here are the bits I thought were worth sharing.

I'm in Boston tonight, in town for Software Development Best Practices. I'll be giving a talk on Wednesday, along with Alistair Cockburn, on "Creating Change One Tic-Tac At a Time". The idea for this talk grew out of a conversation that Alistair and I had back at the Jolt Awards in March and it incorporates information and ideas from a wide range of sources. For my part I'm drawing on The Moral Animal for the importance of status, Taking Charge of ADHD for the idea of a token economy, Flow for insights into what people consider rewarding, Influence for some "weapons of persuasion", not to mention a host of others I can't name, plus my own experiences leading development teams and our experiences at Agitar helping our customers adopt developer testing. These last two categories are probably the most important, because that is where I've come to believe that cultural change is simply the most difficult task anyone can undertake, and changing the practices of a development team in any significant way require a change of culture. Alistiar has a similar view on the importantance and difficulty of cultural change and brings his own diverse and illustrious experiences to bear on the discussion. If you're at SDBP hope to see you there, but if you're not I'd be interested in your thoughts on the topic (j t f at a g i t a r dot c o m).

I just wrote a page on our internal wiki with our policy for dealing with build failures. We thought others might find it interesting so I am sharing it here (the links will be broken for obvious reasons).

Executive Summary

If the build fails, fix it.

Scott Ambler calls it Green Shift, but it sounds more like Bull Shift to me. (via Jason Yip)

Nine gathered tonight at the Thoughtworks San Francisco office tonight, 4 of whom attended Agile 2006 and 5 of us did not. Things I remember from the conversation, in no order of importance:

• Agile has crossed the chasm, "everyone doing it" was the word on the street
• Success stories even in very large teams/companies
• Wanted more discussion on how to change culture
• Coding Dojo session was excellent and would make for a good future BayXP meeting
• Data no longer a four letter word! (Refactoring Databases)
• Surprisingly valuable discussion by some Microsofties of their TDD Pair Programming Game (complete with phase transition diagram)
• The Open Spaces part was allocated a horrible space -- think low ceilings and harsh lighting -- and this combined w/the relative neglect compared to previous years led to a feeling of low energy...
• ...except the Getting Things Done session got high marks
• Crushing Fear Under the Iron Heel of Action got high marks with Ron and Chet described as living cartoons (in a good way)

I know there was more but I didn't take notes, so that's all you get, unless some kind soul adds more info in the comments.

In early August J.B. Rainsberger gave a webinar on TDD for J2EE:

Test-Driven Development is often introduced through simple examples, but many developers would rather dive into the deep end. This free webinar is for those people. J. B. Rainsberger, author of "JUnit Recipes," will show you architecture and design strategies to make it easier to "test-drive" J2EE components. You'll learn how to build a J2EE application while following the cardinal rule of Test-Driven Development: Never write a line of production code unless somewhere, a test has failed.

The recorded webinar is up on the Agitar website linked from this page. Registration is required but if you'd rather not register try out bugmenot.com.

Greetings,

We are proud to announce that registration is now open for the next Continuous Integration and Testing Conference, CITCON London 2006 on October 6 & 7. Space is limited to the first 120 registrants and attendance is free. Registration is on-line.

The conference will be following the same Open Spaces (or unconference) format as the very successful Chicago edition back in April 2006. The Chicago CITCON drew an enthusiastic group of practitioners, all looking to share what they knew and to learn what they could.

We would appreciate your help in getting the word out. Please blog/post/email/chat about CITCON London 2006!

We hope to see as many of you there as can make it.

Sincerely,

Paul Julius
Jeffrey Fredrick
Chair-people, CITCON London 2006

There are many reasons why you might not want to fix a failing test right away. Maybe it's an acceptance test for a feature that you haven't written yet. Maybe it's a regression that it's just not practical to fix right now.

But what to do with that failing test?

If a little bit of feedback is good, a lot would be even better, right?

Is there such a thing as too much feedback?

XP doctrine says that you should stop tracking metrics once they have served their purpose. You should only have 3 or four "Things To Focus On". There's a reason for that.

James Shore is involved in what seems like an great opportunity for computer science students at Portland State University to learn some real world sklls. Along with Dr. Andrew Black, PSU Professor of Computer Science, they are putting on the course Extreme Programming: Principles & Practices. If you're up in Portland, Oregon it looks worth checking out...

It is critical to remember that Dos Equis Driven Design (XXD) is not about the beer. (I'm not saying there was no beer involved, but that isn't the point...)

I got fed up with updating the 20 step instructions on our wiki for configuring a new cruise control machine so I wrote a script and checked it in to CVS. Obvious really.

Despite releasing CruiseControl 2.5 little more than 2 months ago we are looking to get 2.6 out in July. The minor (2.6) vs. trival (2.5.1) version bump reflects some big changes:

• replace Xalan with Saxon. this should both increase the speed of the web application and reduce if not eliminate the dreaded OutOfMemoryErrors.
• use an Ant-style launcher instead of setting the classpath in the CC start script or in the manifest.

With these changes we know there will be issues to shake out so we're seeking alpha testers willing to have a go with the latest and greatest and give us feedback on the mailing list.

Are you up for it?

ps: As a side-effect of the switch to Saxon CC versions 2.6 and later will require JRE 1.4 or later (though you can still build projects under 1.3).

It's like what Whitehead said about notation: "By relieving the brain of all unnecessary work, a good notation sets it free to concentrate on more advanced problems, and in effect increases the mental power of the race."

Outstanding post, Brian. I always wondered what the little star was for in GMail. Now I know. Your post has a little gold one next to it.

Hi, I'm Karl Pauls and my team runs Agitator on a Bonsai / Tinderbox system. I'll be sharing the continuous test and continuous integration techniques that we use.

To get right to it, here is our basic Agitator / Tinderbox setup courtesy of our Product Manager, Nicole Pauls:

Aaron Rhodes has posted a good message detailing how to integrate test results from Agitator into the CruiseControl html email and the web reporting application on the Agitar forums over here. The directions detailed are of obvious interest to people using Agitator but perhaps less obviously can serve as an example for other people looking to add custom information to the CC feedback. Enjoy!

Mocking a singleton is relatively easy.

To begin with, singleton's are all alone in the world; don't be afraid to point this out. Also, most singletons are created, get this, not even in a factory, but merely in a factory method. Finally there is the point that they are little more than dressed-up globals, and IME experience cause more heartache than they're worth. You can point to the many nasty screeds about singletons there are out on the net. All in all, mocking it shouldn't be too hard. A word: if your singleton has unusually large body parts, stoop to insulting that. Sometimes, to be kind, you have to be cruel.

CruiseControl 2.5 has just been released and it has two huge bug fixes:

• Updated Xerces and Xalan jars to fix memory leak (CC-426)
• ProjectWrapper: Fix problem where forcing a build while build is running will effectively kill the worker thread leading to all projects queued but not building (CC-431). Patch by John Lewis.

Along with those bug fixes there are a few new plugins:

• ClearCaseBaselinePublisher: New publisher for use with ClearCase UCM (CC-414). Submitted by Kevin Lee.
• LDAPMapper: Email mapper that uses an LDAP lookup to determine user email addresses (CC-413). Submitted by Esa Laine.
• Maven2SnapshotDependency: New sourcecontrol (CC-438). Contributed by Dan Rollo with fixes by Jerome Lacoste.

So what are you waiting for? Go get it...

Next week Agitar is hosting Ted Husted of the Struts development team (and iBATIS and MyFaces and Jakarta-Commons) in a webinar on What to Do if Your Code Has Few, If Any Tests?. I'm curious to hear what Ted has to say but his talk illustrates the kind of trade-off that exist as our company grows. One the one hand we can having interesting speakers and topics like this, but on the otherhand we're now scheduling the webinars for a global audience, so the scheduled times (7 am and 5 pm PDT) probably work better in just about every other timezone than this one. (</whine>)

Brian Marick says that tests are not specifications but I believe there is a more fundamental distinction.

In math geek terms, specifications are universally quantified statements, ones of the form "for all inputs such that <something> is true of them, <something else> is true of the output." Tests are constant statements, ones with no variables. They look like this: "given input 5, the output is 87."

The main focus is fixtures but it also has some helpers for creating continuous integration scripts. This was all designed around subversion post-commit hooks and there is even a handler for you to run a SimpleXMLRPCServer.

It doesn't matter if you call it Refrigerator Code or Girl Code or Beautiful Code or simply Clean Code, the meme is out there that is isn't enough to say the code works and then leave it at that. Even if you find yourself unmoved by Kathy's aesthetic arguments you should weigh Uncle Bob's assertion that "keeping your code clean is not just cost effective; it’s a matter of professional survival."

If you ever feel yourself drawn toward writing a static method, obey Kevin's Maxim: "in an object-oriented language the simplest thing that could possibly work is an object."

If you're going to be at the Better Software Conference & Expo (and why wouldn't you?) and you want to be infected with the Continuous Integration meme then do stop by my talk titled The Power of Continuous Integration with Automated Unit Tests on Thursday June 29th.

Agitator is not a test generator. It is an exploritory testing tool for developers. You turn to it to answer the questions "what does this code actually do? what did the author forget to consider?"

He's actually practicing two of the best practices I know of: regular one-on-ones and code reviews. In my own management career I found it very hard to stick w/the weekly one-on-one schedule even though I believe it is hugely important. And code reviews are probably the best sofware development practice that (virtually) nobody does. This guy is obviously working very hard to put theory into practice and for that he has my respect.

Kathy Sierra's Clicker trained by our email blog entry just closed a synaptic loop for me. When I give my CI/CA talk I try and make the point that developers quickly become addicted to the positive feedback of that "build successful" email, that even this little reward, given quickly enough after the behaivor you want to encourage, is enough to reinforce the habit. When I spoke to Alistair Cockburn about this at the Jolt Awards he even used the phrase clicker training to describe it. But now reading Kathy's entry something new springs to mind -- in that same CI/CA talk I make the point that the real Return On Investment in developer testing comes when the test fails... and that's an intermittent reward!

So if you want to adopt developer testing as part of your standard practices make sure you're using Continuous Integration. In the beginning it is the "build successful" messages you'll be looking for, but for the long term you'll be hooked on those times the lamp goes red.

Maybe you missed my talk yesterday or maybe you just want more; either way you'll want to check out Paul Dzilenski's Creating an Eclipse IDE Command Line Interface today at 1:36 p.m. in the Theater.

This is the only thing that has ever worked for me.

1. Test your own code as well as you can
2. Next time someone checks in a slap-your-forehead bug, show them the test that would have caught the bug if only they had run the tests.
3. Show them how to run the tests themselves

It won't be long until everyone will want to have tests.

Most developers will not make the investment until they have seen proven returns. The management challenge is to find the early adopter with the courage and vision to take that first step.

Bruce Eckel weaves together the two hot topics in blogland this week. One is the Humane Interface vs Minimal Interface discussion which pits Java's List API against Ruby's. The other is the "Death of Java" topic.

Unlike Jeff, you won't catch me shamelessly plugging my EclipseCon talk on The Humble Plugin.

Keith Ray says utility functions are a code smell and I wholeheartedly agree.

BTW I often use the Adapter Pattern to wrap a class that seems to need utils until eventually the Adapter grows into a whole new class of its own. I have done this to add utility functions for parsing and xpath querying to org.w3c.Document for example.

Back in August I wrote a pictorial guide to creating a headless hello world eclipse plug-in; now I've proposed to cover the same information in a short talk at EclipseCon 2006 and I've decided to use my massive blog fanbase ("hi mom!") to shill for votes. If you're interested in the content (or just an adoring fan) you can vote for me here (registration required to vote).

Oh, and if you're really really interested in learning more about this topic than can be covered in 5 - 10 minutes (the duration of a short talk) then you might be interested in voting for Paul Dzilenski's Creating an Eclipse IDE Command Line Interface long talk proposal.

Happy voting!

The Pragmatic Programmers recommend learning a new language every year. Not because you need to know a whole bunch of languages, but because other languages use idioms that you might not think of using in your everyday language.

There is a discussion on the JUnit list about whether coverage tools are valuable.

I ran a coverage tool on a project of mine last night and found that almost all the coverage gaps were in boilerplate code. An interface required me to return false in a whole bunch of classes.

The duplication was already bothering me. The duplication plus coverage gaps bothered me enough to extract a common base class. Coverage was back up to almost 100% and I liked the new design better.

Go figure.

What’s that? It’s code that you’re so proud of that you want to take it home and hang it on the refrigerator, right alongside of your children’s drawings.

That suggests that a specification should not be written to a consistent level of precision. Precision is needed only where disputes have already occurred or are likely.

Every now again, I hear of a wacky design idea or a challenge to a deeply held opinion on design.

Last night I needed to categorize all of the JVM opcodes according to their effect on the stack. Since there are around 200 of them, it seemed like it would be a tedious task. Fortunately, the JVM spec is online in an editable format. I thought, "Maybe I can parse the opcodes out of the spec and then put them into a format that I can use to build my categorization automatically."

Tom Ball has an interesting blog entry on a good use for finalizers. Library designers can use it to let users of their library know when they have not cleaned up a resource.

The comments on the entry are enlightening. One comments that this could slow down garbage collection. Another comments that they always make sure that any code in their library that has system resource handles manages the closing of that code. I have used this approach in the past, but it was somewhat ugly.

...one major advantage of TDD is the ability to answer the question "how will I know I am done?". We can aswer that question because we have a test that will fail if we are not done and pass if we are.

I have been consulting for a number of teams that have adopted Agile Methods, including TDD. One common issue I have found is that developers drop the discipline of TDD in the face of schedule pressure. "We don't have time to write tests" I hear them say. Before I comment on the absurdity of this attitude, let me draw the parallel. Can you imagine an accounting department dropping dual entry bookkeeping because they've got to close the books on time? Even before SARBOX such a decision would be such a huge violation of professional ethics as to be unconscionable. No accountant who respected his profession, or himself, would drop the controls in order to make a date.

Malcolm Gladwell, in Blink, tells a story about how some students were asked to evaluate their teacher after a semester.

Just to make it easy for any potential candidates out there, here's my whole interview question for the J2EE position.

I want you to build a Hello World application in J2EE. I want you to do the simplest thing that can possibly work.

1. If you are interviewing for a company that makes tools for developer testing you should probably know something about developer testing.

Tight feedback loops provide excellent learning environments. Bacteria learn to beat an antibiotic in a decade or two. But it will be quite a while before dogs evolve their way around the doorknob.

Late followup, but JavaOne was a lot of fun. My sessions went well and I got to attend several enjoyable sessions as well. Thanks to everyone who came to my talks.

For the talk about plugin writing, you can download the slides here

Read the original blog entry

I wrote this article for the March 2005 issue of Better Software. Now available for the first time online... enjoy!

I have worked with many testing common organizations where a common pattern is repeated over and over. It goes something like this — we realize there is more manual testing to do than time available, we decide to automate the testing, and we begin working on a test harness. Several weeks later, we have the start of a harness, but it’s barely useful and we still have not written any tests. At this point, we’re behind — so we abandon the harness and revert to manual testing.

Full article [PDF]

I have these rules that I use for unit tests, primarily because I encounter so many teams that start writing end to end tests, call them unit tests, and give up because "testing takes too long". To me, a test is not a unit test if:

1. It talks to a database
2. Communicates across a network
3. Touches the file system
4. You can't run it at the same time as any of your other unit tests.
5. You have to do special things to your environment to run it (like editing configuration files).

Tests that do those things are okay, but to me they aren't unit tests, and they should be segregated from true unit tests.

We will test everything that could possibly break. We will have tests at the customer level as well as at the programmer level. We will write our tests before we write our code, possibly moments before. We will automate our tests and provide a binary "pass/fail" mechanism for evaluating the result.

I would like to introduce myself as a new Agitator. I am Bob Evans and I am working on some of the next product features for the Agitator.

On Thursday May 26 Jon Udell interviewed Alberto Savoia. Alberto had just been selected by InfoWorld as one of the 25 Top CTO's in the US in 2005. Jon asks Alberto about his vision for Agitator and Alberto demos Agitator extensively.

You can see the replay now if you don't mind registering. If you do, come back in a few days and we'll point you to an unguarded version.

Your Developer Testing writers on the cover of Software Development!

Of course we were incredibly proud winning the Jolt Award by Software Development Magazine - but to be selected for the cover page was even better!

To be sure - the team that has created Agitator and is driving the innovations for the coming releases is (by now significantly) larger than the group on this cover page, but the contributors to Developer Testing were selected to represent the team.

Oh, and Alberto wants to be sure it is mentioned that he is not as small as he appears on the picture - he was sitting on a chair in front of the rest of the group. These vain CTO's!

Agitar has posted the videos from the November 17th Developer Testing Forum--they are now online at www.unikron.com/agitar1/. Presentations include Kent Beck, Sriram Sankar (Google), and an Expert Panel (Russell Gold, Oracle; Rob Mee, Pivotal; Sri Muthu, Wells Fargo; David Vydra, Testdriven.com).

XSL is one of those technologies that I need to interact with often enough that I care about it but not frequently to be good at it. Today while looking for something else -- anyone out there know if there is such a thing as a "breaking non-whitespace" in html? -- I came across this very useful page, which is part of XSL FAQ for Mulberry Technologies' XSL mailing list that Dave Pawson maintains. I get a kick that such serendipity can help me find something so useful, and that by posting about it I can both ensure that I can find it again when I need it and make it more likely for other people to find it. Aren't virtuous circles pleasant?