Wednesday, February 25, 2009

Unit Test Coverage

S.Lott writes in his blog about unit test code-coverage: how much is enough?

Effective tests should account not only for code paths, but also input values and other application state or external environment that may affect the behavior.

For example, it may be easy to get 100% code coverage from tests for a function like the following:
divide(x, y) { return x/y; }
But unless you test for division-by-zero (when the parameter y is zero), you haven't tested sufficiently.

The code-coverage metric doesn't reveal when you've tested a good variety of input values. It only tests if your tests have visited the given lines of code, not what values were in each variable at the time.

Likewise for other application state besides input parameters. Values in other application objects, the contents of databases or files, or the operating system environment can all affect the proper functioning of a class or function that you're testing. These variations are not measured by code-coverage metrics.

It could be argued that if you're testing for external state, you aren't doing unit testing by its strict definition; you're doing functional or system testing. Nevertheless, most people rely chiefly on unit testing tools, because automated unit testing tools that generate code-coverage metrics are pretty easy to use.

While it's a worthwhile goal to try to get high code-coverage in your unit-testing, a score of 100% doesn't guarantee that you've tested enough. Likewise, a score below 100% isn't necessarily an indication of inadequate testing. Code-coverage is therefore not a goal in itself; it's one way of measuring one type of testing.

Friday, February 06, 2009

How Do You Reward Good Clients?

A user recently asked:

I find when I get a 'good client' things go so much smoother on a project (there even seems to be less bugs - weird?). I have a habit of rewarding good behavior from anyone (even if its just a simple thank you).

I am interested to know what sort of things you guys do, and even how you feel about good client behavior.

It would be nice if "good clients" were simply "normal clients," and bad clients were those you avoid working for.

If you want to give them a more explicit and material reward, you could give them a "good customer" discount on the next project. My invoices specify a penalty for late payment, so I suppose you could also offer a small discount (2%) for extra-prompt payment.

Probably the best reward is to give them your best value for their dollar, and continue doing business with them. Work with them with equal respect, communication, and commitment to quality and value. Plus occasional free support, or advice on projects you're not actually working on for them, etc.

It goes both ways. They know when they've got a good contractor who delivers quality work on time and charges fairly. They want to continue the relationship when they get good results. So a good customer won't quibble about nickel-and-dime line items on invoices, pester you about delivery dates, or question your technology choices. They'll accept your rates as economical if you actually give them good work, instead of skipping to another consultant who charges less but does poor work. They'll also refer other clients your way (and probably not the ones they know will be annoying for you).

Really bad clients will get a bit more of a cold shoulder:

  • "I'd really like to bid on your new project but I'm swamped with other work this quarter."
  • "That'll be a rush order so I'm going to have to charge you a premium."
  • "Sorry, I was traveling for a few days and didn't get your RFP."

Unfortunately, bad clients are probably least able to read between the lines when they get this sort of message. It's much easier for them to believe that all contractors are difficult to work with, than to accept that it's they who are difficult.

I'm posting to my blog the questions I've answered on StackOverflow, which earned the "Good Answer" badge. This was my answer to "How do you reward your clients for good behavior?"