Lean Software Factory

<div dir="ltr" style="text-align: left;" trbidi="on"><br /><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US">This post is a follow-up to an invited talk that I delivered at </span><a href="http://www.cesames.net/soiree-thematique-du-20-mars-2012-6844"><span lang="EN-US">CESAMES</span></a><span lang="EN-US">. That presentation, which is available on the box.net widget on the left, talked about complexity, architecture and lean software development. It started from the question – its title – “<i>How to design information systems</i>?” and ended with an introduction to </span><a href="http://informationsystemsbiology.blogspot.fr/2011/11/lean-it-devops-and-cloud-programming.html"><span lang="EN-US">lean software development</span></a><span lang="EN-US">. Today I’ll go further and explore the relationship between the (software) product and the (software) factory. The key idea is derived from </span><a href="http://en.wikipedia.org/wiki/The_Toyota_Way"><span lang="EN-US">the Toyota Way</span></a><span lang="EN-US">: <b>you need a great factory to manufacture a great product</b>. Not surprisingly this will lead to the concept of “Lean Software Factory”. This post is organized as follows. The first section talks about what makes a good software product. I will focus mostly here on software development, but most of what will follow is equally relevant to information systems. The second section deals with the concept of “</span><a href="http://en.wikipedia.org/wiki/Software_factory"><span lang="EN-US">software factory</span></a><span lang="EN-US">”. Looking at the “software factory” means taking into account the software manufacturing process into account, and not only the final product. It is also a metaphor that carries the need for automation, engineering methods and industrial discipline. The last section is about “<b>lean software factories</b>”, which add agile development practices and wisdom to the strengths of software factories, together with a continuous improvement and culture geared towards excellence that is inherited from <i>lean management</i>.<o:p></o:p></span></div><div class="MsoNormal"><span lang="EN-US"><br /></span></div><div class="MsoNormal"><b><span lang="EN-US" style="font-size: 14.0pt; mso-ansi-language: EN-US; mso-bidi-font-size: 10.0pt;">1. What makes a good SW product?<o:p></o:p></span></b></div><div class="MsoNormal"><b><span lang="EN-US" style="font-size: 14.0pt; mso-ansi-language: EN-US; mso-bidi-font-size: 10.0pt;"><br /></span></b></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US">There are many expected qualities and assessing a piece of software is a multi-dimensional tasks. Here I will only focus on three of them, because they are both very popular (all books and papers related to software engineering discuss those three) and very important to the value that is produced through a software product:<o:p></o:p></span></div><div class="MsoNormal"><span lang="EN-US"><br /></span></div><div class="MsoNormal"></div><ul style="text-align: left;"><li><a href="http://en.wikipedia.org/wiki/Modularity" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Modularity</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;"> – “the degree to which a system’s components may be separated and recombined” (Wikipedia) … or the capacity of the architecture to maximize in its decomposition the independence of subcomponents. Modular systems are easier to maintain and to evolve. In the world of software, modularity translates into the minimization of “impacts” when something must be changed.</span></li><li><a href="http://en.wiktionary.org/wiki/versatility" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Versability</span></a><span style="background-color: white; text-indent: -18pt;"> </span><span lang="EN-US" style="background-color: white; text-indent: -18pt;">/ </span><a href="http://en.wikipedia.org/wiki/Evolvability" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">evolvability</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;"> – “the property of having many abilities”, that is software that can serve many purposes, together with “the capacity of adaptative evolution” - software may evolve rapidly to support new functions. This combination is often coined as “agility” in the software world.</span></li><li><span lang="EN-US" style="background-color: white; text-indent: -18pt;"><span style="font-size: 7pt;"> </span></span><a href="http://en.wikipedia.org/wiki/Openness" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Openness</span></a><span style="background-color: white; text-indent: -18pt;"> </span><span lang="EN-US" style="background-color: white; text-indent: -18pt;">: for a piece of software, being open may mean many things. It may means that it exposes API (<i>application programming interfaces</i>) so that other software components may interact with it. It may also mean that the code is open to all for scrutiny and collaborative improvement, as in “open source”. It may mean that the piece of software is built as a <b>platform</b>, which is designed to be enriched by subcomponents written and proposed by other developers. Writing software as a platform is at the heart of successful development strategies, such as Google’s or Facebook’s.</span></li></ul><br /> <div class="MsoNormal"><span lang="EN-US">These three qualities form a deeply interrelated network (each influences the other). They are also notoriously difficult to achieve, especially in a rapidly changing environment. These qualities are resulting properties of both:<o:p></o:p></span></div><div class="MsoNormal"><span lang="EN-US"><br /></span></div><div class="MsoNormal"></div><ul style="text-align: left;"><li><b style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Software architecture</span></b><span lang="EN-US" style="background-color: white; text-indent: -18pt;">, that is, design properties which explain how the software product is organized. This part is well understood, albeit difficult: this </span><a href="http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471228869.html" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">the art of software architecture</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;">. If I follow </span><a href="http://en.wikipedia.org/wiki/Fran%C3%A7ois_Jullien" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">François Jullien’s</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;"> distinction between </span><a href="http://books.google.fr/books/about/A_Treatise_on_Efficacy.html?id=OS_YqZhWuAsC&redir_esc=y" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Greek and Chinese strategy</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;">, architecture is clearly the Greek, top-down, left brain, goal-oriented part of the strategy for producing modular, agile and open software.</span></li><li><b style="background-color: white; text-indent: -18pt;"><span lang="EN-US">The way the product is made</span></b><span lang="EN-US" style="background-color: white; text-indent: -18pt;"> (the software development culture). This translates into “genes” which are embedded into the software product. According to the same distinction, software development culture is the Chinese, bottom-up, right brain, emergent strategy for producing modular, agile and open software. This is less understood and even less formalized, it is a form of wisdom transmitted in an oral tradition. It is also controversial since many still believe that once the specification is completed, it doesn’t matter who writes the code …</span></li></ul><br /> <div class="MsoNormal"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaQcd52wS-vkfUl_7eDwbmhp-eCx58VDfcb2-sugzA9wIsDBSdJIUgZN4bUEQkRMbhMZi9gH_y_DbWnJJYpHkFc0SIhNZZOsETOO1tCZSjtV-vyV5c6QxquIGaTLILKDXID7fQASLHOPQ/s1600/Urbanisation4e.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaQcd52wS-vkfUl_7eDwbmhp-eCx58VDfcb2-sugzA9wIsDBSdJIUgZN4bUEQkRMbhMZi9gH_y_DbWnJJYpHkFc0SIhNZZOsETOO1tCZSjtV-vyV5c6QxquIGaTLILKDXID7fQASLHOPQ/s200/Urbanisation4e.jpg" width="140" /></a><span lang="EN-US"></span></div><div style="text-align: justify;"><span lang="EN-US" style="background-color: white;">I am certainly not one of them </span><span lang="EN-US" style="background-color: white; font-family: Wingdings;">J</span><span lang="EN-US" style="background-color: white;">. On the contrary, I have found over the years that <b>great software products most often come from companies with great software cultures</b>. It is not a new thought either, even for me. In my first book “<a href="http://www.dunod.com/informatique-multimedia/systemes-dinformation-et-reseaux/strategie-et-systemes-dinformation/manageme/urbanisation-soa-et">Urbanisation, SOA & BPM</a>”, I noticed that modularity and agility are properties that describe the whole lifecycle and not only the development part. Agility in code writing does not do much for you if you cannot run your tests in an agile mode (Chapter 6). Similarly, modularity must translate into modular testing (running fewer tests), modular deployment (deploying fewer components) and modular maintenance (fixing problems faster because the area of investigation is limited to a smaller area).</span></div><br /> <div class="MsoNormal" style="text-align: justify;"><span lang="EN-US">The remainder of this post is devoted to « <i>the way software products are built</i> », for lack of time but mostly because my previous writings and books have been about software and information systems <i>architecture</i>. You may read the CESAMES slides to get an overview on achieving modularity, agility and openness through architecture. For those of you who don’t read French, here is a “101” summary:<o:p></o:p></span></div><div class="MsoNormal"><span lang="EN-US"><br /></span></div><div class="MsoNormal"></div><ol style="text-align: left;"><li><span lang="EN-US" style="background-color: white; font-family: Symbol; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><a href="http://en.wikipedia.org/wiki/Model-driven_architecture" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Model-driven Architecture</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;"> : architecture should be based on models and part of the development should be derived with semi-automated methods. Models are another form of “genes” for a software product: careful design at the model stage goes a long way to embed “future-proofness” (using scenario/whatif methods)</span></li><li><a href="http://www.1keydata.com/datawarehousing/data-modeling-levels.html" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Data models</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;"> are critical: from semantics, conceptual analys, ontology to object lifecycle management. Data architecture is the most useful tool to prevent data distribution difficulties which plague most large-scale, heterogeneous information systems.</span></li><li><a href="http://hendryluk.wordpress.com/2009/08/17/software-development-fundamentals-part-2-layered-architecture/" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Layered Architecture</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;"> : defines abstraction levels and reduces the management complexity, a time-proven “good practice” of software architecture.</span></li><li><a href="http://informationsystemsbiology.blogspot.fr/2009/05/soa-tale-of-two-cities.html" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Service-Oriented Architecture</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;">: defines the service structure so that reuse and sharing is made easier. SOA is also the proper tool to align the different stakeholders so that software fits the enterprise’s strategy. A service architecture is a reification of functional architecture (a service may be seen as the combination of a function – what it does- an interface – how to invoke it – and a contract – so-called “non-functional” properties –), which embodies abstraction and encapsulation.</span></li><li><a href="http://en.wikipedia.org/wiki/Process_architecture" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Process Architecture</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;"> : defines the “composition grammar” through a recursive/fractal analysis of interactions. Process architecture is to “Business Process Engineering” what SOA is to “writing down a list of exposed services”: there is much more than listing and formalizing, it is about identifying patterns and sub-patterns, and extracting those patterns which will prove stable through time. It is more an art than a science.</span></li><li><a href="http://en.wikipedia.org/wiki/Event-driven_architecture" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Event-Oriented Architecture</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;">: identifies “events” as pivots for interaction (which is a subpart of process architecture and a foundation for SOA). </span><a href="http://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">Publish/Subscribe</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;"> is another time-proven pattern for building modular designs, which is built on top of a shared/structured event catalog.</span></li></ol><br /> <div class="MsoNormal"><br /></div><div class="MsoNormal"><b><span lang="EN-US" style="font-size: 14.0pt; mso-ansi-language: EN-US; mso-bidi-font-size: 10.0pt;">2. Software Factories<o:p></o:p></span></b></div><div class="MsoNormal"><b><span lang="EN-US" style="font-size: 14.0pt; mso-ansi-language: EN-US; mso-bidi-font-size: 10.0pt;"><br /></span></b></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US">Focusing on the way software products are build leads logically to the concept of “Software factory”. The following illustration shows three way of looking at a software system (or an information system):<o:p></o:p></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><br /></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="background-color: white; text-align: left; text-indent: -18pt;">The first step (left-most) is to look at the technical system (leading to what we call IT : Information Technology). This view is concerned with practical components such as databases, software code and computing resources. Processes are defined in the “computer science sense”.</span></li><li><span lang="EN-US" style="background-color: white; text-align: left; text-indent: -18pt;">Then we step back and add the human into the picture, mostly the user but also other stakeholders (management, administration, strategy,…), leading to Information Systems. </span><a href="http://blogs.cio.com/chris_potts/information_systems_versus_information_technology_how_much_do_people_care" style="background-color: white; text-align: left; text-indent: -18pt;"><span lang="EN-US">IT/IS distinction</span></a><span lang="EN-US" style="background-color: white; text-align: left; text-indent: -18pt;"> is well established in information systems literature. This view adds – rightly so – usage as a key dimension (following the very profound statement that a software application that is not used is of little value), and shared enterprise artifacts such as models and business processes.</span></li><li><span lang="EN-US" style="background-color: white; text-align: left; text-indent: -18pt;">The topic of this post is to step back another time and look at the “production process” (the “factory”) that continuously builds and update the information system. On the right-most part of the illustration, we see both the information systems and the organization (usually the ISD – Information System Division in a large company) that builds and runs it. I have separated the part that builds (dev: software development) and the part that runs (op: operation) as a forward reference to </span><a href="http://en.wikipedia.org/wiki/DevOps" style="background-color: white; text-align: left; text-indent: -18pt;"><span lang="EN-US">devops</span></a><span lang="EN-US" style="background-color: white; text-align: left; text-indent: -18pt;">.</span></li></ul><br /> <div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinOq-RpWb-HvrEphU4-KBH_KdMnmAsdY9fldI9k7C6zjFhLE1BiQbnu2obL7RormrCp8D8jewKdWPPBC0ZBYguNYKAt8R-jpKrGTlJAARgmhlowQ2lcOAzbT7jpIp2xQw_0GGr0x7RWGg/s1600/SoftwareFactorySchema.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinOq-RpWb-HvrEphU4-KBH_KdMnmAsdY9fldI9k7C6zjFhLE1BiQbnu2obL7RormrCp8D8jewKdWPPBC0ZBYguNYKAt8R-jpKrGTlJAARgmhlowQ2lcOAzbT7jpIp2xQw_0GGr0x7RWGg/s640/SoftwareFactorySchema.jpg" width="640" /></a></div><div class="MsoNormal"><br /></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US">Because a software factory handles ideas and digital information, it could be argued that it is an </span><a href="http://en.wikipedia.org/wiki/Autopoiesis"><span lang="EN-US">autopoietic system</span></a><span lang="EN-US"> (this is not a car factory, there is endless supply of ideas and disk space to write new code). Autopoiesis represents the capacity of a living system to continuously develop itself and to transform the process that creates the system and which is a part of the system itself. Autopoiesis generates emergence (of properties) and (what may be seen as) auto-organization. This thread of thought is quite interesting, because it helps understanding the emergence of our three desired properties, however I will leave it to another day, to avoid adding confusion to conceptual complexity.<o:p></o:p></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US">To return to more practical grounds, the goal of a software factory is to deliver “good software products”, rapidly and efficiently. Because we now have the advantage of having read “</span><a href="http://theleanstartup.com/"><span lang="EN-US">The Lean Startup</span></a><span lang="EN-US">” (if you have not, I would urge you to do so, it’s the best way to understand what is truly expected of a great software factory), we know that there is another dimension to SW factory performance: <b>the ability to listen carefully to the product’s users and react rapidly to their expectation</b>. This is why </span><a href="http://en.wikipedia.org/wiki/Eric_Ries"><span lang="EN-US">Eric Ries</span></a><span lang="EN-US"> advocates for </span><a href="http://en.wikipedia.org/wiki/Minimum_viable_product"><span lang="EN-US">MVP</span></a><span lang="EN-US"> (Minimum Viable Products): the SF (software factory) must be able to crank up simple products very rapidly, which are exposed to customers, and modified according to their reaction (this loop is shown on the illustration).<o:p></o:p></span></div><div class="MsoNormal"><span lang="EN-US">A lot of things are known about how to build a “good” SF. Sharing a reference with the previous post, I picked the following recommendations in Curt Hibb’s book “</span><a href="http://shop.oreilly.com/product/9780596517311.do"><span lang="EN-US">The Art of Lean Software Development</span></a><span lang="EN-US">”:<o:p></o:p></span></div><div class="MsoNormal"><span lang="EN-US"><br /></span></div><div class="MsoNormal"></div><ul style="text-align: left;"><li><span lang="EN-US" style="background-color: white; font-family: Symbol; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="background-color: white; text-indent: -18pt;">“Practice 0” : Source Code Management and Scripted Built. This is where the word “factory” becomes self-evident: one must use tools (such as source code management) and industrial discipline to manufacture software. There is a “Maslow pyramid” of skills necessary to reach continuous deployment, scripted built is the first step.</span></li><li><span lang="EN-US" style="background-color: white; font-family: Symbol; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="background-color: white; text-indent: -18pt;">“Practice 1” : Automated Testing, because testing is crucial for quality, as in any industrial process (automated testing means more testing), but also because we want to run tests as early as possible, </span><a href="http://informationsystemsbiology.blogspot.fr/2011/11/lean-it-devops-and-cloud-programming.html" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">following the principles of extreme programming</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;">.</span></li><li><span lang="EN-US" style="background-color: white; font-family: Symbol; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="background-color: white; text-indent: -18pt;">“Practice 2” : Continuous Integration, which means that the product is built continuously in an iterative manner, instead of the traditional waterfall model. Continuous integration to perform integration testing as early as possible; it is also a way to “see the product” and ensure that the factory is united on a common goal.</span></li><li><span lang="EN-US" style="background-color: white; font-family: Symbol; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="background-color: white; text-indent: -18pt;">“Practice 5” : Customer Participation, which is what makes the continuous improvement loop that is pictured in the previous figure possible. Customer participation is at the heart of </span><a href="http://en.wikipedia.org/wiki/Agile_software_development#Agile_Manifesto" style="background-color: white; text-indent: -18pt;"><span lang="EN-US">agile methods</span></a><span lang="EN-US" style="background-color: white; text-indent: -18pt;">.</span></li></ul><br /> <div class="MsoNormal" style="text-align: justify;"><span lang="EN-US">There is much more in Hibb’s book, as we shall later see, but this is a set of sound, time-proven practices. Another great source of inspiration for designing a software factory is the </span><a href="http://informationsystemsbiology.blogspot.fr/2011/11/lean-it-devops-and-cloud-programming.html"><span lang="EN-US">previously-mentioned</span></a> <a href="http://devops.com/"><span lang="EN-US">DevOps</span></a> <span lang="EN-US">“method”. Many excellent software companies, such as </span><a href="http://www.fabernovel.com/fr/works/298-facebook-the-perfect-startup"><span lang="EN-US">Facebook</span></a><span lang="EN-US">, are making references to DevOps, which is not a coincidence. Following </span><a href="http://blog.octo.com/tag/devops/"><span lang="EN-US">Octo’s lead</span></a><span lang="EN-US">, I would summarize DevOps’ contributions with two principles: “Infrastructure as code” and “Continuous deployment”. </span><a href="http://www.agileweboperations.com/the-implications-of-infrastructure-as-code"><span lang="EN-US">Infrastructure as code</span></a><span lang="EN-US"> is the ability to address all computing resources (either cloud computing or a regular data center) through API and software. It follows the principles of “Practice 0” (tools & automation) and give true flexibility and agility to the software factory. </span><a href="http://blog.xebia.fr/2012/01/09/mise-en-place-d-une-organisation-devops/"><span lang="EN-US">It also changes the “ops culture”</span></a><span lang="EN-US"> and prepares for the next step. </span><a href="http://en.wikipedia.org/wiki/Continuous_delivery"><span lang="EN-US">Continuous delivery</span></a><span lang="EN-US"> is the ultimate step of continuous development/testing/integration/deployment. It helps shortening the continuous improvement loop from Eric Ries and is necessary to move towards a “lean software factory” as we shall see later. </span><a href="http://royal.pingdom.com/2010/06/18/the-software-behind-facebook/"><span lang="EN-US">Taking a look at Facebook development and operations’ culture is worth the effort</span></a><span lang="EN-US">. Facebook has developed large-scale continuous deployment into an art. I recommend watching Chuck Rossi’s tech talk on “</span><a href="https://www.facebook.com/video/video.php?v=10100259101684977&oid=9445547199&comments"><span lang="EN-US">Pushing millions of lines of code five days a week</span></a><span lang="EN-US">”.<o:p></o:p></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><br /></span></div><div class="MsoNormal"><b><span lang="EN-US" style="font-size: 14.0pt; mso-ansi-language: EN-US;">3. Lean Software Factory<o:p></o:p></span></b></div><div class="MsoNormal"><b><span lang="EN-US" style="font-size: 14.0pt; mso-ansi-language: EN-US;"><br /></span></b></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US">The idea of a “lean software factory” comes from mixing “lean software development” (cf. <a href="http://informationsystemsbiology.blogspot.fr/2011/11/lean-it-devops-and-cloud-programming.html">previous post</a>) with “software factory”. That is, adding to what we just saw a few other principles which are taken from Agile methods and from the Toyota Production System (cf. <a href="http://shop.oreilly.com/product/9780596517311.do">Hibb’s book</a>). This does not mean that all projects from a lean software factory should follow a SCRUM method (take as an illustrative example of agile method). Although this is debatable, I believe that some projects are still better run with a classical V-cycle model (<a href="http://blog.ippon.fr/2012/04/25/scrum-breakfast-lagilite-chez-google/">I am not not the only one</a>). There is no magical formula to define the “SCRUM/ agile boundary”, but V-cycle still makes a lot of sense for “back-office projects” with stable and complex specifications (such as a billing system). On the other side, agile methods are better suited to front-office projects (when the human in the loop plays a bigger role), where detailed requirements are difficult to produce because the need is ill-defined or changing rapidly.<o:p></o:p></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><br /></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US">To keep this post short, I will summarize the concept of “Lean Software Factory” with four “principles”. Each of them would require a full post, which I may do later on – <b>Each “principle” is in fact a set of practices since “<i>lean</i> is something you do, not something you talk about”</b>.  These principles add to the previously developed concept of “Software Factory”. The first principle captures the essence of “agile methods” and, therefore, only applies to agile projects, whereas the rest applies to both forms of software development (that is, including V-cycle development).<o:p></o:p></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><br /></span></div><div class="MsoNormal" style="text-align: justify;"></div><ol><li><span lang="EN-US" style="background-color: white; text-align: left; text-indent: -18pt;">The first principle is to <b>apply an agile method </b>such as SCRUM (Backlog + small batch + test as early as possible + continuous build) whenever the end-user has an important stake with the software product. Agile methods fit nicely with the principles of a SF as described in the previous section. They emphasize customer participation and continuous integration. The lean “touch” adds the concept of small batches (called sprints in SCRUM) and the principle of testing as early as possible, with the goal of producing the right code on the first time. Building a product one small step at a time with regular opportunity to adapt to environment feedback is a key idea which is well developed in “The Lean Startup”. Another key inspiration from <i>lean</i> is to reduce “lead time”, which translates into short sprints but also into time-constrained steps, including design.  This last technique also works within V-cycle: using time-constrained workshops to jointly achieve the requirements and design phases. Those familiar with Google’s software development values will recognize the “lean influence” (develop quickly, only what is necessary, remove waste).</span></li><li><span lang="EN-US" style="background-color: white; text-align: left; text-indent: -18pt;">The second principle is to <b>emphasize autonomous teamwork</b>. Teamwork is critical for all kinds of projects and for most activities. A key teamwork moment from the lean culture is <a href="http://en.wikipedia.org/wiki/Kaizen">Kaizen</a>, which is used to solve problems and search for continuous improvements (two sides of the same coin). Following “the Toyota Way”, all ideas are welcome and everyone’s contribution is useful in a LSF.  There is a lot to be said about kaizen (which I won’t do today) : it goes further than the continuous improvement culture that one may find everywhere (such as the feedback loop in the SCRUM method). “Lean culture” brings a systemic approach to problem solving, including the search for root causes with “<a href="http://en.wikipedia.org/wiki/5_Whys">5 Whys</a>” and the construction/follow-up action plans, such as the famous “<a href="http://www.thequalityportal.com/notes/A3Shook.pdf">A3</a>” from Toyota. Autonomy is a systemic condition for kaizen in particular, but more generally empowerment is necessary to achieve performance.  One of the best book to understand kaizen and autonomous teams is “<a href="http://www.lean.org/Bookstore/ProductDetails.cfm?SelectedProductId=261">The Lean Manager</a>” by F. & M. Ballé.</span></li><li><span lang="EN-US" style="background-color: white; text-align: left; text-indent: -18pt;">The third principle is to <b>leverage visual management</b>, which often takes the form of a project/product room. Visual Management has multiple benefits, the first of them is <i>synchronicity</i>, that is working with the same time. In the world of an industrial factory, the lean concept that embodies synchronicity is “<i>takt time</i>”. Although <a href="http://en.wikipedia.org/wiki/Takt_time">takt time</a> also plays a role in a software factory, synchronicity is mostly about working at the same pace, without waiting. Dynamic wall-planning is the best tool to ensure synchronicity, together with the frequent stand-up meetings in front of the said planning, where the whole team keeps track of progress and difficulties. Visual management is also a tool for systemic training. Toyota encourages its workers to use as many schematics and illustrations as needed to share the detailed functioning of technical processes or pieces of equipment. This translates easily into the world of software. Using the walls to share architectural insights with the team is an efficient way to contribute to continuous learning as well as increase efficiency and prevent errors. Visual management is also used to visually control “work in process” (<a href="http://en.wikipedia.org/wiki/Work_in_process">WIP</a>). Reducing WIP is another key tenet from <i>lean management</i>that applies very nicely to software development. The practice of visualizing WIP with small cards (post-it) that are pasted on the wall according to their status in the development process is called <i>Kanban</i>in the lean IT world, in reference <a href="http://en.wikipedia.org/wiki/Kanban">to the Totota pull system</a>. Kanban / JIT (just-in-time) in the world of software is not as strictly enforced as in the world of industrial manufacturing. Still, the “pull” chain of control, where every step in a process is “serving” the following step (that is, providing what is needed at the right time, according to the availability of the receiving step), is a great tool to increase efficiency and avoid “overload” of tasks that are waiting to get completed. The “kanban” practice has two benefits: making each member of the team aware of the availability of others who will use their work later on, and reducing overloads and piles of unfinished work through an implicit capacity limit to what is accepted into the development process. The emphasis is on “no waiting” to both improve agility and quality (asynchrony is the best way to lose “context” when the control is handled from one player to another and to produce misunderstandings – this is explained in <a href="http://www.dunod.com/entreprise-gestion/entreprise-et-management/strategie-et-politique-de-lentreprise/processus-et-entreprise-20">my last book</a>, and is especially true for software development, which is a complex task, and where one quickly looses focus once your work  is completed.</span></li><li><span lang="EN-US" style="background-color: white; text-align: left; text-indent: -18pt;">The last principle tells that a Lean Software Factory is a place with “<b>love for the product</b>”. This supposes two things: to know, understand and “love” the customer, on one hand, and to love software code on the other hand. The love of code is another multi-dimensional mix of culture and practice.  The practice part draws on extreme programming (code reviews, peer writing) and software factory (cf. previous section) with a “lean touch”. Mary Poppendieck explains how the <a href="http://en.wikipedia.org/wiki/5S_(methodology)">5S</a>of lean may be applied to the practice of source code management. Lean practice also translates in writing less code (see previously mentioned Curt Hibb’s book). The culture part is also a mix of aesthetic (writing elegant, minimal code that one is pleased to show to the other members of the team) and discipline (following coding styles and guidelines, appropriate density of comments, etc.). This is very close to the values of open source software, one could say that a lean software factory should write its code as if all of it was going to be open source. We are getting back to the opening argument in the introduction: designing open software is a state of mind, hence it should be supported by the culture. There are two reasons why “the love of code” is emphasized as a key value. The first one is that application of what is learned about Toyota: to build a great company, the employees need to love the product that they are making. For Toyota it is about cars, in a software factory, it is about code. Lean management is all about professional pride, technical skills and craft. A lean software factory is not about undistinguishable “man-days” that may be outsourced to a lower-cost country. The second reason is that code is a “live object”, with a renewal rate which is getting higher as the world becomes more complex (<a href="http://blog.ippon.fr/2012/04/25/scrum-breakfast-lagilite-chez-google/">Petra Cross from Google says that 50% of their code changes every month</a>). Code that grows iteratively needs to be re-factored regularly (iteration yields accumulation); like a garden, it needs constant care and attention.</span></li></ol><br /> <div class="MsoListParagraphCxSpLast"><br /></div><div class="MsoNormal"><br /></div></div>

Lean Software Factory

This post is a follow-up to an invited talk that I delivered at CESAMES . That presentation, which is available on the box.net widget on the...

Read more »