Test-driven information

Lean software: ne vous trompez pas de cible

2011-03-23T14:50:00.001+01:00

Au détour de mes lectures sur divers blogs et des échanges avec mes pairs au sujet du Lean Software Development, je trouve qu'il se répand une idée reçue assez dangereuse: faire du Lean signifie éliminer les gaspillages. Dans les faits cela se traduit souvent par une chasse aveugle et impitoyable au moindre gaspillage. Les résultats vont parfois du désastre à souvent une amélioration molle et peu durable. Pourquoi ?

A tous ceux qui pensent que Lean signifie supprimer les gaspillages, je souhaite suggérer un retour aux fondamentaux, c'est à dire au monde de l'automobile d'où nous vient le Lean. Certes, dans un précédent post je me gaussais des parallèles périlleux entre domaines d'ingénierie, mais je ne suis plus à une contradiction près. Et puis comme le souligne le commentaire de PIF, il faut savoir cross-fertiliser, si je peux me permettre le néologisme.

Mura, muri, muda
Fondamentalement donc, la "pensée lean" consiste à éliminer 3 facteurs d'inéficience: le mura (surcharge), le muri (variabilité), le muda (gaspillage volontaire). Dans une lettre datée du 6 Juillet 2006 (traduite en français), Jim Womack nous éclaire sur la façon d'aborder ces facteurs. NB: Jim Womack fait partie des 3 rédacteurs du livre fondateur du Lean The Machine That Changed the World. Jim nous suggère d'adresser d'abord la variabilité et la surcharge, et ensuite le gaspillage volontaire. Il confesse lui-même avoir initialement abordé ces facteurs à l'envers. Son constat est sans appel, si vous commencez par supprimer les gaspillages, vous allez générer de la variabilité et de la surcharge de manière incontrôlée et par conséquent ré-introduire d'autres gaspillages ailleurs dans votre système par effet de compensation. C'est un cercle vicieux. Le message est donc clair: maîtrisez la variabilité et la surcharge dans votre processus, ensuite diminuez ces facteurs à mesure que vous éliminez les gaspillage. C'est la meilleure façon d'implémenter des améliorations durables.

Qu'est-ce que la variabilité dans l'ingénierie logiciel ? Les livraisons sans cesse décalées, des petites livraisons suivies de très grosses releases suivies de petits patchs, etc.

Qu'est-ce que la surcharge ? Des équipes qui travaillent 12h par jour, les week-end d'astreintes, des serveurs de dev en permanence au taquet, etc.

Agilité, agilité, agilité

Je ne vois qu'un moyen simple d'éliminer ces 2 facteurs variabilité et surcharge: passer à l'agilité. Que vous soyez Scrum, XP, Kanban, ou autre, vous livrerez des petits lots de tailles similaires avec des échéances fixes, cela adresse pour partie le problème de variabilité. La surcharge est adressée par les valeurs de ces différentes méthodes, il faut évidemment une certaine discipline pour éviter le surengagement, mais la diminution de la variabilité permettra de réduire l'effet de "bourre" avant des grosses livraisons.

Un conseil en forme de conclusion: ne faites pas de Lean Software Development sans passer par l'agilité, les résultats risquent d'aller à l'encontre de vos objectifs. Et vous, quel est votre expérience du Lean Software ?

Le goût des autres ... domaines d'ingénierie !

2011-02-15T09:13:00.000+01:00

C'est le buzz du moment: le software craftsmanship est sous toutes les plumes des bloggers agiles qui se respectent, notamment là ou là. Et je ne vais pas me priver de relayer le buzz car d'une part je me respecte (ouf), et d'autre part le sujet semble être hautement polémique et donc j'aime ça.

Il est toujours amusant de voir les réactions vives et parfois étayées de semi-insultes que provoquent ces discussions. L'arrivée de l'agilité avait donné lieu à des diatribes enflammées dans la blogosphère. Je trouvais à l'époque qu'un profil particulier se mettait régulièrement en lumière dans ces échanges: l'ingénieur en bâtiment refoulé. Celui qui voulait utiliser la même méthode pour construire un immeuble que pour construire un logiciel. Je n'ai pas encore compris pourquoi. Un exemple ici dans le commentaire n°9. NB: il y avait aussi la catégorie ingénieur civil refoulé qui voulait construire des ponts comme des logiciels. Les plus audacieux d'entre eux n'hésitaient pas à clamer vouloir construire une centrale atomique de la même manière qu'un logiciel ! Si si, je vous jure, c'était dans un commentaire sur le numéro 1 (le seul numéro d'ailleurs) du Valtech Mag en Décembre 2006, dommage qu'il ne soit plus en ligne, ça devrait passer à la postérité ! Bref, l'ingénieur en bâtiment refoulé avait raté sa vocation et était arrivé dans l'ingénierie logiciel avec l'ambition ferme d'y mettre un peu d'ordre et de relayer ses développeurs au rang d'ouvrier, ses concepteurs au rang d'architectes, et ses leaders techniques au rang de chefs de chantier. Malheureusement l'agilité compromet ses plans et on ne l'entend plus tellement de nos jours. NB: évidemment il va être le premier à me laisser un commentaire.

Mais voilà, le software craftsmanship a réveillé une autre catégorie d'ingénieur : les industriels malheureux. Un exemple ici (Gregory, sans rancune). Porté par la vulgarisation des termes "industrialisation", "usine logiciel", et "lean software development" qu'on utilise à tours de bras de nos jours dans l'informatique, l'industriel veut croire qu'un logiciel est fabricable en grande partie mécaniquement, depuis les plans de son concepteur ou selon un processus immuable, et avec une intervention réduite à la portion congrue "d'opérateurs" (autrement dit peu qualifiés). Il semble oublier que le développement informatique n'a pour but que de produire une pièce unique qu'on fait évoluer au cours du temps, et pas des milliers, voire des millions de pièces identiques, et que donc les procédés, méthodes, outils, ou pratiques utilisés pour produire la pièce N, ne sont plus tout à fait applicables pour produire la pièce suivante. Le modèle de l'industriel est la fabrication d'automobiles, un monde idéal qui doit nous montrer l'exemple. On y trouve pourtant des faillites vertigineuses, des ratages monumentaux, et des loupés en tout genre (même pour les meilleurs), dont fait rarement état l'industriel malheureux lorsqu'on en vient à comparer l'automobile et le logiciel.

Personnellement j'ai hâte de voir arriver les prochaines vagues d'ingénieurs malheureux qui vont nous expliquer comment développer du logiciel en se calquant sur les autres domaines d'ingénierie. Ainsi on pourra peut-être entendre:

l'ingénieur agricole qui va nous expliquer comment labourer et fertiliser des systèmes d'exploitation avant d'y planter des graines de code qui vont pousser lentement chez les hébergeurs pendant l'hiver et donner leurs fruits d'expérience utilisateur au printemps. L'amélioration d'un SI passera par des boutures systématiques des vieux systèmes pour donner naissance à de jeunes applications que l'on pourra éventuellement greffer entre elles afin d'obtenir des systèmes plus productifs et de meilleure qualité, le tout abondamment arrosé d'engrais générateur de code,
l'ingénieur en biologie qui va nous expliquer comment procéder à des tests cliniques sur des bouts de code souche pour évaluer les effets d'une crème de refactoring avant de l'appliquer sur des applications vivantes en production,
l'ingénieur chimiste qui va nous expliquer que comme rien ne se perd et tout se transforme, si vous voulez dissoudre votre ancien SI dans une solution acide de nouvelles technologies, les émanations de gaz de changement vont fortement incommoder vos utilisateurs. Attention, si ce gars là devient copain avec l'ingénieur agricole, les effets risquent d'être désastreux sur votre SI et contaminer progressivement votre infrastructure avec du code "générétiquement" modifié,
etc.

Bref, sortons-nous de ces comparaisons fatigantes avec les autres domaines d'ingénierie et continuons de réfléchir à comment faire les choses dans notre propre domaine. Les sources d'inspiration sont certes bonnes à prendre mais ont leurs limites qu'il faut savoir accepter. Le mouvement du Software Craftsmanship est une étape importante dans notre réflexion qui a le mérite de replacer les connaissances techniques au coeur du processus de création d'un logiciel. Elément qu'on a tendance à oublier avec la généralisation des méthodes agiles.

Un dernier point qui me chiffonne dans ces discussions autour de notre profession: pourquoi nous généralisons souvent à partir de l'observation de notre propre nombril ? J'avais traduit il y a bientôt 9 ans un article très intéressant de JoelOnSoftware qui distinguait les différents mondes du développement logiciel. Il est toujours tellement d'actualité ! Dans toutes nos discussions sur le software craftsmanship nous réduisons systématiquement nos constats à notre propre monde. C'est un peu comme comparer le job du plombier et du menuisier. Ils ont bien sûr des points communs, de là à appliquer les mêmes règles à l'un et à l'autre, le raccourci est hasardeux...

Stratégie d'automatisation des tests: un filet de pêche ou une fusée ?

2010-09-08T23:31:00.019+02:00

Jusqu'à récemment j'avais l'habitude d'utiliser la métaphore du filet de pêche pour illustrer la stratégie d'automatisation de tests que l'on applique le plus couramment sur les projets agiles. Cette stratégie est illustrée par la pyramide de Mike Cohn: une forte base de tests unitaires, un étage plus restreint de "spécifications exécutables", et un petit chapeau de tests "bout-en-bout". La métaphore du filet me vient d'un article que j'ai lu il y a quelques années *, qui parlait plutôt d'un filet de sécurité, et que j'ai tenté de m'approprier et remettre à la sauce pêcheur. Ça donne quelque chose comme ça:

"Une stratégie d'automatisation de tests est comme un filet de pêche, plus le filet est grand et ses mailles petites, plus la capacité à attraper des poissons est forte. NB: pour les esprits les moins vifs, les poissons représentent les anomalies. Le filet se doit d'être suffisamment solide pour permettre une utilisation pérenne, et suffisamment bien attaché au bateau pour ne pas perdre sa récolte.

Les tests unitaires représentent les petites mailles de votre filet à attraper des anomalies. Il en faut beaucoup pour couvrir une surface suffisante, et il faut qu'il soit bien unitaire pour que la maille de base soit suffisamment fine, sinon on laisse passer les petits poissons. Ce petit maillage est fait d'un matériel souple qui s'adapte à son environnement: les courants, les bancs de poissons, la résistance de l'eau. Le test unitaire l'est également: les outils et les pratiques (TDD) doivent permettre de le faire évoluer facilement dans son environnement (le code) . Tout comme les petites mailles d'un filet de pêche, un test unitaire peut "casser" sans remettre significativement en question votre capacité à attraper les poissons, il y a beaucoup d'autres mailles dans votre filet. De plus il est facile à réparer pour peu que le pêcheur consciencieux le fasse rapidement.

Ce filet de petites mailles nous paraîtrait presque suffisant. Mais voilà, va-t-il résister aux gros poissons ? Que se passe-t-il si un thon ou un requin blanc se pointe ? Et si l'on accroche le fond ? On risque de perdre tout simplement un gros morceau de son filet, ou d'en jeter une partie parce que "ça ne sert plus à rien". Il faut donc renforcer ce filet avec une structure plus rigide, fait d'un maillage plus large mais dans un matériel plus solide. Cette structure représente les spécifications exécutables. Les tests y sont d'ordre fonctionnel ou métier, ils sont moins fins au regard du code, mais plus solides car ils portent sur des vérifications plus stables dans le temps (le métier). Le code est lui soumis à un remaniement incessant et les vérifications des tests unitaires sont donc moins stables. Le métier, lui, ne change pas tous les jours, le cycle d'évolution est plus long. Néanmoins, le matériel utilisé reste souple pour ne pas subir négativement les contraintes de son environnement. Il faut également trouver le bon dosage de ce maillage afin de ne pas alourdir inutilement notre filet mais de le rendre suffisamment solide. De la même façon les tests des spécifications exécutables doivent représenter les exemples clés, qui facilitent la compréhension des exigences.

Alors, a-t-on maintenant un filet de pêche suffisant ? Non bien sûr. Mais que lui manque-t-il pour être utilisé ? Des fixations pardi! Des câbles, des cordes, des poulies, bref des attaches qui lui permette d'être relié au bateau et d'être manipulé dans son environnement cible, la mer! Ces attaches sont faites d'un matériel lourd et rigide et paraissent solides à première vue, mais elles sont aussi fragiles que les petites mailles car elles centralisent l'ensemble des contraintes de l'environnement. Ces attaches sont les tests de bout-en-bout automatisés. On teste ici l'application déployée dans son environnement (presque) réel. Ces tests ne sont pas nombreux mais indispensables. On teste ici ce qui ne peut pas l'être dans les 2 maillages précédents, comme l'interface graphique ou la performance par exemple."

Finalement je suis revenu à quelque chose de plus simple il y a quelques jours lors d'une soutenance: une métaphore basée sur le principe de la fusée à étage. C'est plus proche de la pyramide et cela s'explique plus rapidement que le filet de pêche. Chaque étage contribue à la mise en orbite du satellite, mais chacun a un rôle bien précis que les autres ne sauraient endosser. Plus on monte dans les étages, plus sa fonction se sophistique et se rapproche de l'objectif final. J'aime également le fait que les étages de la fusée servent directement un objectif métier, la mise en orbite du satellite, alors que le filet de pêche sert plutôt un objectif compensatoire à un système défaillant, la pêche aux anomalies. C'est plus proche de la vision moderne des tests que j'essaye de transmettre.

Et pour vous, l'automatisation des tests est-elle un filet de pêche ou une fusée à étage ?

*Brushing Up On Functional Test Effectiveness, Jennitta Andrea, Better Software Magazine Nov/Dec 2005

Webcast des MS Techdays 2010

2010-05-23T10:57:00.013+02:00

Les webcast des derniers MS techdays sont disponibles depuis quelques semaines. On y retrouve notamment la table ronde à laquelle j'ai participé. Il y avait du beau linge autour de cette table, qui ressemblait d'ailleurs plus à une brochette d'experts étant donnée la configuration des tables :

Laurent Bossavit, membre du bureau de l'Agile Alliance et Agile France,
Véronique Rota, ex-Valtech et auteur d'un livre référence sur le management agile,
d'autres anciens Valtech comme Xavier Warzee, l'animateur Microsoft de cette session, et Jean-Laurent Fabre de Morhlon, team leader chez Vidal,
des experts de sociétés de conseil, comme Eric Mignot, coach agile chez Pyxis, et Luc Legardeur, fondateur du French Scrum User Group et de Xebia,
et donc moi, en tant que coach en transformation agile chez Valtech.

Le webcast de la session s'apparente plus à un podcast car seuls les slides de la session sont visibles, slides qui n'étaient là que pour appuyer les changements de sujets. Il faut donc reconnaître les voix des uns et des autres. Cette session fût enrichissante au niveau des sujets abordés et des questions posées. L'agilité reste un paradigme qui soulève encore beaucoup de scepticisme et de blocages. Malgré tout, on sent bien que la révolution est en marche et que la plupart des organisations participantes se demandaient "comment y aller" et non plus "est-ce qu'il faut y aller".

=> Voir le webcast

Suis-je un mauvais coach agile ?

2010-02-10T23:31:00.019+01:00

C'est la question que je me suis posé en lisant un n-ième article de blog américain sur le coaching agile avec, en vrac, 4 citations de bouquins faisant "autorité", 3 références à des modèles comportementaux, 9 utilisations de termes japonais, 2 métaphores sur les arts martiaux, et 1 signature prétentieuse. Je me suis senti quelque peu "nanifié" de ne pas connaître la moitié des références exposées.

Car oui, je clame haut et fort mon ignorance:

je ne connais pas le modèle de Tuckman
j'ai fait du judo quand j'étais jeune et je ne vois aucune similitude avec le développement informatique
shu ha ri est un concept vague dont j'essaye de percer le sens lorsque je propose des sujets à des conférences agiles
Kano est le nom du restaurant en bas de chez moi
je n'ai jamais réussi à lire un livre sur l'agilité jusqu'au bout; le seul que j'ai terminé concerne le Lean (The Machine That Changed theWorld) et ne parle pas du tout d'informatique

Une telle étendue d'ignorance et de mauvaise volonté devrait me conduire droit à l'échec sur mes missions de coaching agile.

Et pourtant ...

... mes différentes missions de coaching agiles se passent bien et atteignent leur objectif (jusqu'à preuve du contraire). Je vous propose donc mon top 5 de ce que je mets en oeuvre dans mon coaching d'équipe agile (garantie sans termes japonais):

pas trop de changement d'un seul coup: ce qui compte c'est l'application des 4 principes agiles, le reste c'est de l'enrobage à distiller au compte-goutte et dont une grande partie relève de l'initiative de l'équipe. Si certaines techniques ne sont pas bien appliquées dès le départ ce n'est pas grave, il faut se concentrer sur les quelques éléments clés qui caractérise une équipe agile. A mon sens ce sont les itérations en time box, les démos avec du logiciel qui fonctionne pour de vrai, et la communication directe. Il y en a bien sûr beaucoup d'autres, mais cela vient progressivement,
du pilotage opérationnel: il faut que l'équipe, et plus largement mon client sache où il va et quels sont les mécanismes de gestion d'un projet agile. La période de coaching ne doit pas générer d'effet tunnel. Par exemple, quitte à me brouiller avec des agilistes chevronnés, je fais en sorte que les itérations atteignent leurs objectifs, et surtout la première itération. L'échec est un luxe que se permettront les équipes rompues à l'agilité. Egalement, il est utile d'organiser des comités de pilotage du projet, celui sur lequel intervient l'équipe coachée, à chaque fin d'itération et avec des participants qui vont au delà du PO (typiquement la hierarchie). C'est là qu'on présentera les indicateurs, les projections de budget à terminaison, la variation du périmètre, bref les mécanismes de pilotage. C'est en quelque sorte la démo de la mission de coaching agile.
de l'écoute, de l'écoute, de l'écoute, et encore de l'écoute: il me semble très important de prêter une oreille attentive à tout le monde c'est-à-dire bien sûr l'équipe, le Scrum Master, et le Product Owner, mais aussi son chef, les collègues, les utilisateurs, le commanditaire de la mission de coaching, son chef, les instances transverses, bref tous! Je ne jette aucune information ni n'écarte d'un revers de la main les réactions inappropriées. Tout commentaire est bon à prendre, toute réaction a son fondement et cache parfois des informations capitales pour la réussite de la transition vers l'agilité
de la communication: expliquer en permanence les principes de l'agilité à toutes les parties prenantes (direction, organisations transverses, équipes métier, contributeurs, prestataires, etc). On s'aperçoit vite que l'équipe coachée est entourée d'un nombre insoupçonné de personnes qui ont un rôle à jouer ou qui influencent d'une façon ou d'une autre le travail de l'équipe
de l'empathie: les membres de l'équipe coachée vont traverser un changement parfois très profond de leurs habitudes de travail. Cela apporte son lot d'incertitude, d'angoisse, voire de souffrance dans certains cas. Je ne suis pas là pour torturer mes clients mais pour leur faciliter la tâche, j'essaye de ne jamais l'oublier.

Bon, tout ça pour dire quoi ?

Pour dire que je suis prêt à entendre qu'il faut sans cesse se cultiver pour améliorer ses pratiques, mais il faut pouvoir ... mettre sa culture en pratique!! Je trouve qu'il y a quand même un bonne partie de toute l'agitation autour du coaching agile qui tient de la démonstration mathématique voire parfois de la masturbation intellectuelle, et qui ne sert pas à grand chose concrètement.

Sayonara !
NB: ce message ne s'adresse pas aux bloggeurs pragmatiques qui adressent également tous ces concepts dans leur messages. D'ailleurs, j'envoie mes remerciements à JC qui reste ma première source d'information pour comprendre le baragouinage de certains :)

Dojo TDR @ XP Day France 2009

2009-06-10T14:17:00.015+02:00

This is a late feedback on my session:

+ good attendance, room was full but not overcrowded, great :)

? this was my fifth experiment of this type of dojo, we started the first experiment with a colleague last November; I still had high expectations on the output

+ the subject (Blackjack) was an excellent base to work on, thanks to Gojko whom I stole the idea from

+ thanks to the XP Day goodies, I got a much better timer than I used to have for timekeeping

- I proposed the fitnesse wiki to record the tests, not a very efficient tool for authoring, but that's all I had on my laptop

+ the audience was very dynamic trying to help the keyboard holder

? I started the 5mn turns, intentionally choosing a test that was not at the heart of the problem, just to check whether the participants come back to the core topic, they did not, so I'm not sure that was a good idea

- the participants started quite late to identify interesting tests, actually that was half-through the session when I asked them whether the tests roughly illustrated the Blackjack rules

+ towards the end, one participant got rid of FitNesse and grabbed a pencil to draw some tests on the paperboard, that helped a lot to share an understanding on blackjack rules

+ the conclusion that I made to the audience was something like: "if you want to learn about the domain and the requirements, don't pay too much attention to the rules, they're distracting you from the objective"

+ finally, I received very good feedback about the session from the participants, if you attended the workshop and have some more feedback, please let me know

Outils de tests open-source

2009-05-14T22:50:00.004+02:00

Les slides de ma présentation au dernier afterwork Valetch sont sur slideshare:

Outils de tests open-source

Qu'avez vous testé aujourdhui ?

2008-12-08T23:04:00.005+01:00

C'était aux Valtech Days en Novembre dernier, vous pouvez retrouver la présentation en français et en vidéo sur Valtech TV. J'ai mis la présentation sur slideshare:

Qu'avez vous testé aujourdhui ?

What have you tested today ?

2008-10-04T13:51:00.012+02:00

I will be presenting a lightning talk at the Valtech Days 2008 in Paris, the 21st of October.
It will be a "refactored" version of the lightning talk I made at the aa-ftt workshop the 4th of August just before the Agile 2008 conference. There is a low-fi shooting of this talk on google video, thanks to Elisabeth Hendrickson:

One of the attendees also wrote a blog post referring to the talk.

Since then, I went on thinking more and more on how to present these values and the rationale behind them. One of the consultant in Valtech also found a pretty good name for this talk: "What have you tested today ?". I think this question nicely wraps up the 3 values that I emphasize. There is the name of my lightning talk! Come the 21st of October to know more.

The material of the TDR workshop at Agile 2008

2008-08-09T14:53:00.016+02:00

I'm posting here the (electronic) material that I created for my workshop at Agile 2008, Test-Driven Requirements: Beyond Tools. Feel free to reuse it to play the game yourself, I just want you to keep me informed of the output.

Game 1:
I created this game after an experiment in which I participated at the University of Linköping (Sweden) back in 1999. I found the concept interesting enough to make a parallel with software engineering. I named the 2 main players "Product Owner" and "Developer" and I added a couple of roles, the User and the Tester, to add a bit more spice to that game and make it closer to software product development process.
To play the game, you need 2 identical sets of wooden blocks, a table, and a splitter.

Here are the instructions of the game:
- Audience instructions
- Players instructions

Game 2:
I completely made up this one (or think so until someone tells me it's an old game they played at Agile 1975 ...). In this game, groups of 3 to 9 persons try to uncover the requirements of a vague specification that a user (one of the group's member) had read just before. The objective is not so much to uncover real requirements or needs, but rather to use tests to explore the needs and formalise a specification. The users don't know anything concrete, so the specification should be elaborated as a result of a group collaboration.

The game starts with a short presentation reminding the audience about tools and techniques for using tests as specification. Then, one user in each group is given a Memo to read to get to know the context of a system on which the group will work. I allow the group to work 20 minutes and then ask them to write the result of their collaboration on a sheet of white paper. I will post soon the output of the workshop (I have to upload a report on the Agile2008 submissions website).

Agile 2008: Presentation of Examples stage

2008-07-31T22:24:00.003+02:00

Here is a podcast where Adam Geras presents the Examples stage of Agile 2008. He talks about my session towards the end of its speech. Thanks Adam.

You will also find podcasted presentations of the other stages here.

Preparing Agile 2008: TDR workshop

2008-07-29T20:04:00.002+02:00

Yesterday evening my colleagues and I organized at Valtech office in Paris a rehearsal of my workshop for Agile 2008. This event was open and announced on the XP France web site. My colleagues took this opportunity to videotape the session and we invited a production company to do the same, so hopefully we should post a video abstract in a couple of days on the Valtech website. I would like to thank all the participants that took time to come after the standard office hours. They've been greatly rewarded with pizzas :)
Eric has already posted a summary of the evening (in french).

It was the first time I conducted the second part of the workshop, as I'm already conducting the first part in the TDR courses. This session uncovered lots of interesting points.

About the format and organization first. It lasted approximately 135 mins despite my efforts to shorten some phases, so it is still 45 mins too long. I can to optimize time by writing guiding sheets for the participants so I don't need to explain too much what they're exepected to do, that will also make the instructions clearer. I will also reduce the number of wooden blocks so the first part of the workshop can fit within the expected 30 mins.

Regarding the content I was surprised how unnatural it seems to use tests for exploring needs and formalizing requirements. The participants did use examples in their discussions, but it seems they didn't realize that these examples were already test embryos, so they took note of generic software requirements made after the examples, and then wrote tests from those requirements... Well, exactly the opposite stuff I wanted to see... So I encouraged the groups to work on concrete examples, and start writing tests from those examples.

Then going from examples to concrete tests seemed another uneasy step. I observed that many of the participants proposed to write tables (FIT habits ?) where I felt writing text and progressively maturing a DSL would make more sense and be more efficient for the exercise. As a result, participants were spending lots of time trying to put everything in a couple of tables and thus looking for suitable formats rather than simply writing tests as examples unfold. I also saw the teams progressivelly overwhelmed by the complexity of the requirements and losing track of the basic examples they had discussed at the very beginning of the exercise. They did not want to write tests until they get a sufficient idea of the requirements and they understood their complexity.

This workshop tought me that using tests for exploring needs and specifying software isn't that obvious. We should definitely think of some communication techniques so that teams get some guidance in this field, something like the TDD process applied to requirements (well, we've come a long way, it is the idea of test-driven requirements after all...). TDD does not allow you to write bunch of tests at once and then produce the code, it tells you to do one thing at a time, so you're not overwhelmed by the complexity. What could be the process for TDR ? What about: 1. get an example, 2. write a test, 3. get agreement on that test. We need to think about that and I'm really eager to conduct the workshop at Agile 2008 to check the output.

Some words in French about Test-Driven Requirements

2008-06-11T13:29:00.001+02:00

Bent Jensen presented Lean to Valtech consultants

2008-05-15T16:42:00.005+02:00

Yesterday evening we had a presentation of Lean Software Development by Bent Jensen at our Valtech premises in Paris. Bent is helping Elisabeth, a colleague from Valtech, in a mission where our client wants to apply Lean techniques to reduce the cost of their product. Bent has spent a couple of days in Paris and we took this occasion to organize this presentation in the context of our weekly evening training agenda.

Bent made a clear and concise presentation of what Lean is and where does it come from, as well as he explained what Lean actually means in the software development field. After the presentation, Elisabeth talked about their ongoing mission and what they achieved so far, illustrating her explanations with pictures from battlefield.

I really appreciated to hear and discuss application of Lean in software development field. After reading the reference book The Machine That Changed The World, that gave me much more concrete examples and analogies in our field. The point of the presentation that I especially recall is when Bent talked about so-called exploration phase (first iterations) and construction phase (after some iterations) in the lean development process he sketched on one of his slide. He pointed out that this is the way Toyota does in their product development process. Lots of talks are going on in the Agile world whether running an "iteration 0" to settle down some architectural components or frameworks. I think he had an interesting point for this debate. To some extent, what Bent presented recalls a bit the RUP (Rational Unified Process) with its elaboration and construction phases (but no idea here of inception and transition).

Bent's company has a blog where he posts news. I found there some of the stories he told during his talk, like the Lego bug tracking system or his trip to Japan.

(Feed)Back from XP Day France 2008

2008-05-14T16:56:00.002+02:00

I attended the XP Day France 2008 last week, and also gave a talk on the second day. The conference was a success as they recorded more participants than last year. From my point of view I participated in sessions much more interesting than I expected, and especially enjoyed some enriching workshops.

In the morning of day 1, I attended the workshop on Lean System. That was a role-playing game in which a team had to build 10 paper-shadocks in an assembly line-like mode during rounds of 10 mins. After each round the customer (the workshop leader) would accept or reject the finished units depending on what she found acceptable. The project manager would then calculate the financial results based on the sells (accepted shadocks), consumed commodity (paper sheets), team-member wages, facility (tables), etc. I played the shareholder, not a very demanding role as I just had to sit and wait for the financial results, then look very disappointed as the results were quite bad until the last rounds. Before beginning a new round, the team could only take 1 action to improve the output and increase the overall results. Beside being fun, the workshop actually tought some interesting lessons while it was not really focused on software development:

first of all, I realized that Lean System is not mainly about removing waste, it is firstly about removing variability among instances of the same steps. This point, though, might not make much sense in software development and this is probably why we talk much more about removing waste in lean software development,
applying lean system may drive to reduce human resources while increasing productivity and quality. I thought only productivity and quality was the focus while keeping stable resources. The workshop leader therefore stressed that initiating a lean system within an organisation might fail badly if it does not happen in an economically favorable context, because employees will not contribute to a system that might lead to removing their job.

In the afternoon of day 1, I attended the Lightning Talks led by my colleague Yannick. I also made a try at the XP laboratory but soon left the session as it turned too technical to me. What I experienced there was indeed very weird: at the beginning of the session, the organizers explained briefly what was realized thus far and what was the stories they proposed to make in the first iteration of the session. They then asked whether the participants were comitted to delivering these stories by the end of the 10 mins iteration. And guess what, nobody said no ! None of us had a clear idea of the existing design, code quality, or past problems, but we all comitted to deliver the stories. Even more striking is the fact that everyone but me then rushed to the available laptop PC to dig in the code and start programming. Well, I left the session soon after as I'm quite unable to undertsand or program a line of JAVA. I did not feel like that was a great XP application. I however had the occasion to discuss this brief experience with the workshop oragnizers, François Beauregard and Eric Mignot from Pyxis, and it confirmed what I felt: the organizers let the team made some mistakes so that they could learn the XP values. They also were very surprised that no one, whatever the sessions, refused the comittment on the proposed stories having no idea of the design and problems. This is even more striking knowing that this laboratory did not have any financial stakes and that most of the participants are not beginners in the Agile world. That makes me think the way is still long until we master the Agile values and apply them naturally.

Day 1 ended up with a good dinner. I had the occasion to discuss extensively with an employee of Parkeon, especially on their use of FitNesse. I now have better examples to share if I encounter again the objection that FitNesse and TDR techniques are not suited to embeded software (objections that I often hear when I'm teaching the TDR course).

On day 2 I opened the day with my talk on Test-driven Requirements, while 2 of my colleagues also made a presentation (Romain on continuous integration and Nathalie on contracts for agile projects).

I then attended a very enriching workshop about Leadership. The principle was simple, the participants had to build structures with Lego blocks. We run three such sessions, each with a different type of leadership. The first session was driven by 2 leaders with a commanding type of management, the second had no leader and the team built a church in a self-organizing fashion, and the third had 2 coaching leaders (I was one). A global restrospective at the end gave us the opportunity to discuss the characteristics of a good leader and what they missed to do in the exercise. Beyond the lessons that this workshop was supposed to bring about, I realized something more after the retrospective: when there is a leader, everyone holds him responsible for the failure or success of the exercise, on the other hand, when there is no leader, everyone tries to identify what was good and bad with herself or himself. In brief, we are naturally feeling responsible when there is no designated leader while we naturally shift all the responsibility on the leader's shoulders when there is one. What this workshop tells to me is: if you're a leader, expectations will be high on you whatever the type of management you're applying, so you'd better succeed :)

Finally I attended a presentation on practicing the conflict. The presenters made their talk dressed in kimonos and played small funny scenes to illustrate their points. Briefly speaking it was about using an aikido-based technique to master a conflict and not to avoid it. Beyond these techniques, they stated that conflict is a mandatory step towards successful teams, argueing that conflicts can bring about better solutions. Although I understand and share some of their points, I still cannot consider that the conflict is a must have for successful teams. I rather think that confrontation is a must. Sometimes confrontations turn into conflict so we indeed need techniques to master conflicts but I don't see them as a healthy step toward sucess.

Before leaving, I spent some time discussing with the Pyxis guys and getting a demo of Greenpepper (great tool!).

That was my experience of the XP Day 2008 !

Agile 2008

2008-05-13T13:12:00.005+02:00

My session proposal for Agile2008, the world conference about agile software development, has been officially selected ! This is a great news and means I will be flying to Toronto beginning of August to attend this exciting event. I'm scheduled tuesday the 5th at 10.45 AM, I will therefore have the pleasure to kick off the Conference sessions just after the keynote (monday is reserved to research topics and icebreaker). I now have to prepare the details of my session and make sure I can get the material I need to run the workshop smoothly and make it a successful experience for the participants. I will be posting some updates on my preparation as it comes.

Test-Driven Requirements versus functional test automation

2008-04-29T14:03:00.002+02:00

I am updating the case study of the TDR course that I recently developed for Valtech Training. During the first sessions of the course, it turned out that the trainees had a difficult time understanding the differences between Test-Driven Requirements and GUI Functional Tests. So I decided to improve the case study with real examples of automated GUI functional tests.

The case study is a simple webapp that is supposed to be an online banking system (it is of course greatly simplified). I am the product owner of this app, and one Valtech consultant developed it for me (because I can hardly code something now). I built the functional specifications with FitNesse, using test tables to interact with the developer and discuss the functional requirements. In turn, he used the fixtures in a TDD manner to drive his development. Below is an example of the functional requirement that describes an immediate money transfer :

As you can see, we mixed textual descriptions in a use-case style with an example to illustrate the description. The example is actually a test, specifying preconditions, actions, and verifications. We used a RowEntry fixture to specify the preconditions, i.e. to populate the database with test data, then we used an Action fixture to describe the actions, and a Row fixture to verify the results. The example is in french. If you cannot understand french, you just need to know that we spent some time to choose appropriate fixture names, so that the test tables do not feel like automated tests or freaky technical stuff inserted in plain text specifications. We really wanted that the test tables read like real examples, so we chose fixture names like "There exists the following accounts", or "Consult account details", which read more naturally than freaky names we often see with FIT/FitNesse examples "com.myapp.test.fixtures.ConsultAccount". We also put some effort to mature a domain specific testing language (DSTL), so that we could reuse the expressions for precondition or verification with different requirements. For example, we used a lot the fixture "There exists the following accounts" to set up initial test data.

Now let's move to the GUI testing. I made use of a Selenium-FitNesse bridge (the like of webtest) to integrate my automated GUI tests with my specifications. That bridge provides a testing mini-language to allow non-technical people like me writing automated test cases without knowing Selenium scripting. I created a couple of cases to test the webapp, below is an example of an automated test case that traces to the specification given above:

In this example, we see that the flow of actions and verifications is overwhelmed with interface details and strategic instructions (wait for page to reload, etc). The testing mini-language gives a better readability than the equivalent Selenium script would do (some test specialists like to call this "keyword-driven testing"), but still, the test case is not good at serving the specification side. Both types of tests are complementary and should co-exist. The first FIT-like type serves to drive and support the specification process, it also helps in building a regression safety net; the second Selenium-like type serves to test the application and make sure every piece of the software properly fits together. In the end, we should get something like this:

The screenshot above shows the organisation of my FitNesse wiki for the webapp. I created a topmost suite that contains two "sub-suites": one is the specification artefact, containing the FIT-like tests, one is the acceptance test artefact, containing the Selenium-like tests. I made use of links to trace tests from the second artefact to the first, so that I can have an idea of impacted GUI tests when I change the specifications. I can run seperately the specification tests and the acceptance tests, or within a single click. It should be clearer now that TDR and GUI fonctional testing serves a different purpose. I used FitNesse and Selenium because they are open-source, but the same kind of approach can be followed using test management proprietary tools like QualityCenter in combination with test robots and requirements management tools. It's not going to be the same price though.

Next generation testing tools

2008-02-11T19:58:00.002+01:00

During an internal Open Space Technology in Valtech about one year ago (12th Feb 07) we had a discussion on software testing frameworks and test automation tools. I initiated this discussion by proposing a topic about how to make a framework that would hide the programming details of functional test automation and help focusing on writing test cases. We exchanged about existing commercial tools and their limits. We also discussed about frameworks, or wrappers, built on top of those tools to allow for better test creation. Valtech India has built such a wrapper, named LoFat, based on Excel spreadsheets and that can use QuickTestProfessional, Selenium, or SilkTest to execute the functional tests. My current client has also built such a wrapper on top of QuickTestProfessional. We finished the openspace discussion by sketching a simple and theorical architecture to support our approach. After this Open Space, we did not keep this discussion rolling.

Some time ago I came across an article from Jenitta Andrea titled Envisionning the Next Generation of Functional Testing Tools. That's a big piece of work that depicts our expectations towards test automation tools in a bright and comprehensive way. Following this article, Jenitta organised a workshop for the Agile Alliance in October 2007, gathering test automation tools experts to push the discussion further on. There are many reports of this workshop on participants' blogs, and a digest on InfoQ. A discussion group has been created on Yahoo to keep the discussion rolling. I think the consultants that participated in our open space discussion a year ago should subscribe to this group, because there are really interesting stuffs going on. As an example, an interesting web seminars is planned 19th of February, featuring Jenitta and Ward Cuningam, about next generation functional testing tools. Ward might present a new tool from Thoughtworks that supports the approach described by Jenitta. Well, by reading the description of the seminar, there is no question that Test-Driven Requirements and Functional Testing are now one single issue.

The (Toyota) way to go

2008-01-03T11:40:00.002+01:00

Toyota has become the biggest car producer in 2007 overtaking General Motors who had been holding this position for 80 years. This is quite a big news! Toyota has been paving the way to efficient production management for decades, inventing lean and other manufacturing management concepts. Lean can be applied everywere and be transposed to many other businesses. Lots of companies look in this field to improve their process throughput while keeping stable resources.

In software development field, the definitive reference for lean thinking is the work of the Poppendiecks, which is gathered on their website. Many agile development models and practices are either inspired by lean thinking or verify lean principles:

TDD implements a perfect "pull" organisation of the code and unit tests tasks
eXtreme Programming has many practices verifying lean principles (Jacques Couvreur wrote a good article on how XP relates to the Toyota way),
Continuous Integration and software factories are a way to make jidoka happen in the software development process
Test-Driven Requirements (TDR) is an extension of TDD, applying lean principles to the other steps of the software development process (I made a presentation, in French, on this topic during the Valtech Days 2007)
and so on...

Although it seems Agile Values were not directly inspired by Lean Thinking by the time of the Agile Manifesto, there are so many connections today in the software development field that it is difficult to differentiate either one. However, I think that Lean covers a broader area of interest. Software development organisations that seek to be "agile" today will probably seek to be "lean" in the coming years.

Continuous integration and functional tests

2007-12-12T15:53:00.002+01:00

I just wrote an article for my current client, an investment bank, to explain to project managers what it takes to perform non-regression functional tests in a continuous integration process. When I say non-regression functional tests, I mean real functional tests that stimulates the GUI to perform tests. My main point is to argue that continuous integration is not exactly like continuous testing and that their respective objectives cannot be combined as easily as just integrating functional regression tests into the continuous integration process. This is actually what the managers ask for when we start automating the functional tests: "can we integrate these tests into the continuous integration process ?". Well, I could merely answer yes, for it is technically possible, but that answer would not be fair because they cannot integrate the functional tests into their specific continuous integration process. Let me explain this.

Usually the projects we are working with have a continuous integration process including the following steps: compile, build, code check (this step is usually referred to as "test" but I don't like this name), and (sometimes) package. So they are doing continuous integration and they are continuously checking the code, period. Problem is that they want to do continuous testing, I mean not only basic unit tests, they want to continuously perform functional non-regression tests on every build. And this is another story, because it usually requires a fully operational testing environment to test the application at runtime (which is needed for "real" functional tests). Between the time a build is ready for testing and the time it can actually be functionnaly tested, lots of operations are conducted manually, like deployment tasks, database and plateform setup, production data copy, startup of shared services, etc. Some managers don't know that they must fully automate the deployment in testing environment if they want to integrate fonctional regression tests in the continuous integration process. So first point was to stress on fully automating the deployment process.

But this is not the major issue to be addressed. Another point that I stressed was about not breaking the continuous integration benefits. Continous integration is based on quick feedback to the developers: the quicker the feedback, the better the fix. Anything that increases the delay between the time a developer checks his code in the source repository and the the time he gets notified of a failed build jeopardizes continuous integration success. Thing is that functional regression tests is time consuming, and comes after deployment (as noted above) which can be even more time consuming. So it does not look like a good option to automatically perform functional regression tests in the same workflow as continuous integration. Besides this, it does not make sense to functionally regression test every build

The solution that I advised is to create workflows that are triggered subsequently to provide some sort of increasing verification level while not breaking continuous integration. The target organization is to split the job into at least 2 workflows, but that can be more:

the classical continuous integration workflow (compile, build) that is triggered as soon as a new piece of code is introduced in the repository. It is usually extended to check code (unit test, code rules, ...), but I advise to do so as long as it does not exceed 10-20 mins.
It is better to add one more workflow to the process between continuous integration and regression workflow so that tasks like package, generate documentation, install, or even code verification could be removed from the continuous integration workflow and be included in this "intermediate" workflow. It should be triggered as soon as a successful continuous integration workflow has been performed (successful build).
a regression workflow that includes deploying and regression testing (functionnaly) on the test plateform and is triggered at most once a day (say at night), most probably once a week depending on the development cycles and team size.

In short, my answer to the question was:
- fully automate the deployment process
- seperate functional regression tests in another workflow than the classical continuous integration workflow and trigger it at most once a
day

After releasing this article to my client, I started thinking that the name "continuous integration process" as used in the software programming field today is really a bad name. But this is another story.

First post

2007-12-04T11:37:00.001+01:00

This is a simple, yet meaningful title for this post, so that no one will expect too much out of it.

I finally created this technical blog, at least more technical than my personal blog, as an answer to Eric's prayers. I intend to drop here my thoughts and readings on subjects that are of some interest to me (well, this is the point of a blog after all), actually everything that relates more or less closely to software testing, namely:
- functional test automation
- test-driven requirements
- quality management
- continuous testing
- ... and many more topics that I don't even know yet