Mr Gojko Adzic held a two-day seminar for Knowit's customers and employees on Specification by Example in October 2014. Mr Adzic is a strategic software delivery consultant who specializes in agile and lean quality improvement, in particular agile testing, specification by example and behaviour driven development. His book Specification by Example was awarded the #2 spot on the top 100 agile books for 2012 and won the Jolt Award for the best book of 2012. In 2011, he was voted by peers as the most influential agile testing professional. He can be reached by sending email to gojko@neuri.com or visiting http://gojko.net.
Here are some ideas that struck me during those two days and attempts at explaining why I consider them significant.
Mr Adzic repeated several times that "people have a very hard time telling you what they really want but they can easily tell you if you are right or wrong". I seem to recall it wasn't his idea originally but he has put it to good use. Applied to developing software this idea means that the business people have a very hard time expressing exactly what kind of system and with which particular set of features they really want the developers to build them. It is therefore the developers' job to ask them questions like: "Would you find it useful if the system did this and this?", "Is it more beneficial to have this or that feature and in which order should we build them?" and "What should happen when the system gets this special set of input values?".
The most valuable questions to ask revolve around so called edge cases. They reveal the most information of the nature of the system that is being built. An edge case is a situation where the input to the system is valid but special in some way that the system has no obvious response or the currently defined processing rules would not be able to cope with the input. During the seminar we practiced with a few imaginary example systems one of which was a blackjack machine. An example of an edge case with a blackjack game would be the situation where both the dealer and the player get a blackjack. What should the system do? Does the house win or the player? Or should it be a draw and the bet simply returned? Note that this is also a business decision. The business people might want the dealer to win in a draw. On the other hand, they might want the player to win in hopes of attracting more players and thus creating more revenue and ultimately more profit.
These questions cannot be left for the developers to decide, it is not their job. Also, if the edge cases arise only when the developer bumps in to them while coding, it is much too late, let alone if it is the tester who stumbles on them. They must be tackled early on while the business people are available and before any actual coding begins. Testers are extremely valuable here as they have a mind trained for detecting edge cases after spending years analyzing boundaries and otherwise doing their damnedest to break the systems they test. I find this compelling proof for the benefits of including testers in the design of software as early as possible.
One of the many exercises Mr Adzic gave us was to design a small system in groups. Then he set us up by giving us requirements that turned out to be both incomplete and self-contradictory. Groups could have asked Mr Adzic for clarification on the missing and misleading parts but nobody thought of that. The most important lesson here was to realize that this is the case in real life, too. The requirements that developers get from the business people are bound to be incomplete and inaccurate, even downright false. Not on purpose, of course, but nevertheless. Therefore it is the developers' and testers' job to question everything and take nothing for granted. I see this as excellent motivation for asking those questions that need to be asked and lace them with the most useful edge case examples. Ask them often and ask them early.
Apart from learning about the details of the system being built examples also help in documenting the specifications that have been defined. In the seminar materiel there were dozens of small snippets of feature definitions from real life systems. We were asked to evaluate them from three points of view: their usefulness for a developer, for a tester and as documentation. The best specifications were useful on all three aspects and seemed to follow the pattern of describing the what (a description of the feature), the how (e.g. a gherkin style given-when-then definition) and lastly, a few carefully chosen examples illustrating both the happy case and the most important edge cases.
Some specifications listed so many scenarios or examples that they became confusing and the system behaviour wasn't clear any more. According to Mr Adzic, this is usually a sign of a modelling problem and means you have to redesign your specification, perhaps by recognizing and introducing a new variable to break your problem down differently or along a new axis.
When done right this type of documentation is both readable by humans and executable by machines. Documents become tests and tests become documents. After that, there is much less chance of documentation being out of date! Anybody can find out what the system does in a particular situation simply by executing the appropriate part of the specification document that also serves as a test. I haven't come across this kind of a setup anywhere and must confess it sounds a bit utopistic but it is a tickling idea, isn't it?
