250bpm

I have a library in charge of writing video to disk. The entire internal mechanism of the library changed (from fopen() to mmap()) but the API stayed the same. I suspect 90% of the LOC were changed in the whole library…

The unit tests were still valid because the API didn't change. Having the tests in place made me feel confident to replace the guts in a weekend. This was a huge benefit.

"The unit tests were still valid because the API didn't change. Having the tests in place made me feel confident to replace the guts in a weekend. This was a huge benefit."

See, the thing is that you're in a situation where the API is already ossified—you can't change its API very much because you'd break existing clients. Unit tests work great in that situation.

Where they don't work so great is when you still don't have clients and you're trying to come up with a good API. You spend a lot of effort writing a bunch of unit tests… and then you realize your API, which still doesn't have any code using it, is poorly organized. Now you have to throw out all of the existing unit tests.

@Tony: It's external API (the one exposed to the user) that stayed the same, right? That's what end-to-end tests are supposed to test. Unit tests test internal APIs (invisible to the end user).

@Luis: There's in interesting paradox involved here: Unit tests work great once the project is nearing the maintenance phase. There's no much change going on, so the test are never broken by rafactoring or such. At the same time, however, the product in the maintenance phase is already pretty stable and there's not much incentive to write unit tests.

e2e tests would be across contexts. A better debate might be what constitutes a context. This can vary by project. In a typical web app, there would probably be contexts for the UI and the server, and it might be split by functionality or bounded context (DDD). UI unit tests would mock calls to the server as they are not part of the context. Server tests would be at the http level. If you have separate contexts on the server you would want to test the contracts between them. Also, architectural influences may introduce contexts, for example, if you use hexagonal architecture/ports and adapters/DDD, the domain may be considered a context. This is typically because you desire isolation from 3rd party components that may change over time, so your public API would be at the port (ports and adapters) and you would have test implementations of your adapter layers. e2e tests would then be using real implementations. So, in short, public API is influenced by product, technical and architectural concerns.

How is the server API part of the public interface? Public interface is what the end user is using, i.e. the GUI in the case of web app.

Contexts are a business concern. A library would usually be an implementation detail of a particular context, and changing implementation details will not affect tests. In the case of hexagonal architecture/ports and adapters you are making an explicit business decision that you want to de-couple that component from your core domain.

A more concrete example would be a UI, Product, Order and Billing context. Unit tests would be:

UI - Given known response from Product context, does UI display products correctly
Product context - Given a request for products, are the correct products returned
UI - When clicking submit, does the UI send the correct request to the Order context
Order context - Given an order request, is the correct order placed event raised
Billing - Given order placed event is received, is the correct amount invoiced

e2e test would be the whole flow through these.

This is just an example of how you might choose to define your contexts/public interfaces. You could decide that the app is one context, but then you should test everything through the UI. Also, you shouldn't be constrained into thinking that the public API is what the "customer" sees. Architectural quality factors play an important part and testability is an important factor that may lead you to split contexts. A common example of this is splitting a processing pipeline into intermediary outputs. It's all a trade off.

Also, the idea of testing at the public interface is an ideal. Sometimes you might need to be pragmatic and place tests around implementation details, but you should always be aware these tests are brittle and you should be prepared to throw them away.

Yep, I was ranting against testing private interfaces. I think we are in accord here.

Frankly, I don't know. I've never tried TDD myself, so it's hard to say. My feeling though is that TDD may be useful in the cases where the problem is well understood (i.e. when you are writing your 5th system doing basically the same task). In such cases it may be possible to get the unit boundaries right immediately and write the unit tests accordingly.