Software Ecosystems and Application Sustainability

<div dir="ltr" style="text-align: left;" trbidi="on"><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBbP4wnbvFwaE5fbWMRCsuWUNioUjm6fHZf1TxtNWOuzYbAAnJyPJyepgN7mgFUH7qz_6Je9wNc9jFoIH3avYob2NJCtZixEtI0Vf0LP9nwO3NYroGrB0jVYM16Tvt8JcN0hU6qaROtXw/s1600/Keene.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBbP4wnbvFwaE5fbWMRCsuWUNioUjm6fHZf1TxtNWOuzYbAAnJyPJyepgN7mgFUH7qz_6Je9wNc9jFoIH3avYob2NJCtZixEtI0Vf0LP9nwO3NYroGrB0jVYM16Tvt8JcN0hU6qaROtXw/s1600/Keene.jpg" /></a><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Today’s post is a set of simple, yet somehow deep, thoughts about the systemic nature of different ecosystems related to software. I was trained in the 80s to think about software costs in a “traditional software engineering” manner, using KPI, metrics and a spreadsheet (using cost models that were popularized by <a href="http://www.speakers.co.uk/our-speakers/profile/peter%20g.%20w._keen">Peter Keen</a> in the 90s). Somehow, this shows in my <a href="http://www.amazon.com/Information-Technology-Chief-Executive-Organization/dp/1438911114">second book</a>  where one may find references to the classics (<a href="http://en.wikipedia.org/wiki/Barry_Boehm">Barry Boehm</a>, <a href="http://en.wikipedia.org/wiki/Capers_Jones">Casper Jones</a>, etc.) in the bibliography section. What characterizes this way of thinking is that it is a <i>static</i> approach (even if the system changes, one think about a “snapshot” taken at a given time), <i>controlled</i> (one assumes that all stakeholders cooperate under a common control) and <i>global</i> (the cost model operates on what is thought to be a complete picture).<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Life in this century is different when it comes to software. Software is a “live thing” – in the sense that it constantly evolves to adapt to its environment -, mostly distributed, with a large number of stakeholders whose strategy escape the control of the software developer. Hence <i>static</i> should become <i>dynamic</i>, <i>controlled</i> should become <i>collaborative</i> and <i>global</i> should become <i>distributed</i>. The dynamic and complex relationships between the stakeholders and between the various distributed players who contribute to building a software piece yield the “<a href="https://mitpress.mit.edu/books/software-ecosystem">ecosystem</a>” label. This word is borrowed from biology and <a href="http://informationsystemsbiology.blogspot.fr/2013/12/seven-keys-for-complex-systems.html">is a signature of complexity</a>.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This observation is actually one of the reasons for the title of this blog “<b>Biology of Distributed Information Systems</b>”. It helps to think about software and information systems by borrowing concepts for biology and ecology, and it is definitely necessary to switch from a <i>static</i> to <i>dynamic</i> analysis.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This is a first post on this topic, so I will keep things simple (hence somehow incomplete and arguable), and focus on three ecosystems:<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">The OS, platform and application ecosystem</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">The open-source ecosystem</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">The application developer ecosystem</span></li></ul><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span style="font-family: Verdana, sans-serif;">I must apologize in advance to real software experts :). First, this is a post intended for readers with no precise skills nor knowledge about software. Second, I will reason in an “abstract category” way that will not dive into interesting but difficult distinctions. For instance, in this post, an “app” is a piece of interactive content, whether it is a “true application” written for a smartphone, a simple HTML page, an HTML5 page decorated with Java script or an hybrid mobile app. For this first post, I am aiming at a “big picture from 10000 feet up”.</span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"></div><div style="text-align: justify;"><br /></div><span lang="EN-US"></span><br /><h3 style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">1. S</span><span style="font-family: Verdana, sans-serif;">oftware Global Ecosystem </span></span></h3><span style="font-family: Verdana, sans-serif;"><div style="text-align: justify;">The starting point of the argument is the need for software that evolves constantly.  You may accept this at face value because <a href="http://readwrite.com/2007/10/16/the_future_of_software_development#awesm=~oCCRyyaqgZCKuv">it is a commonly heard argument</a>. If you need convincing, the need for constant evolution comes both from the technology (the “what is possible today ?” perimeter <a href="http://mashable.com/2010/12/20/constantly-changing-technologies/">changes constantly</a>) and the users. The complexity (i.e., richness) of software usage today means that the “<a href="https://www.appcelerator.com/blog/2014/04/rutberg-summit/">user is in the driver seat</a>”. That is, software needs to be co-designed with users, which is of the principles of <a href="http://informationsystemsbiology.blogspot.fr/2013/10/lean-startup-lean-innovation-factory.html">Lean Startup</a> (to name one reference, hundreds would apply here). This leads to an incremental model, which in turn requires a (much) faster code production rate. I will assume that you buy this argument, since this is not the topic for this post, and it is a fairly common assumption.</div></span><br /><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">From this we derive <b>two key consequences</b> for software in the 21^st century (as opposed to the century when I was trained as a software developer):<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ul><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Much more innovation is needed</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">, which requires the help of an <a href="http://en.wikipedia.org/wiki/Open_innovation">open innovation</a> model. This leads to the concept of platforms, API and apps.</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Much cheaper software production</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> is needed, which itself requires a new level of sharing/reuse, based on a common/universal software architecture.</span></li></ul><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Software productivity, as defined by the cost to produce a <a href="http://en.wikipedia.org/wiki/Function_point">function point</a>, is improving slowly. This is a topic which I have addressed in depth in <a href="http://www.dunod.com/informatique-multimedia/systemes-dinformation-et-reseaux/strategie-et-systemes-dinformation/manageme/le-si-demystifie">the previously quoted book</a>. I have a vested interest with this question since I started my career as a computer scientist trying to build tools (<a href="http://link.springer.com/chapter/10.1007%2FBFb0013534#page-1">languages</a>) that would increase this productivity significantly. It turns out that the world needed a much more efficient way to reduce software development costs as we just saw and has found it through massive reuse, thanks a to common software architecture :<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ol><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Open OS</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : open operating systems (LINUX, Android, etc.) have become, thanks to open innovation in the form of open source, massive repositories of reusable value. For reasons that will become clear in Section 3, the world needs as few of those as possible.</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Platforms</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">: on top of OS, platforms have emerged. One may think of the most common open source tools such as <a href="http://en.wikipedia.org/wiki/Apache_Software_Foundation">Apache</a>or <a href="http://en.wikipedia.org/wiki/MySQL">mySQL</a>, the <a href="http://www.convinceandconvert.com/social-media-strategy/amazon-google-apple-facebook-and-the-gang-of-four/">GAFA platforms</a> or the web browsers. The rise of platforms over the last 20 years is coupled with the rise of <a href="http://en.wikipedia.org/wiki/Application_programming_interface">API</a>(Application Programming Interfaces) and the associated technologies (XML, Web Services, REST, JSON and the likes).</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Apps </span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">(interactive content) : this is what the end user sees and interacts with. The combination of SDKs and platform APIs, together with open-source libraries, have made the production of apps orders of magnitude more efficient than when I started writing code 30 years ago.</span><span style="font-family: Verdana, sans-serif; text-align: justify;"> </span></li></ol><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4cHt9nBTC02kLbkpc6fxsJIqtBr1dZZy_GS1WYQuQ9Me0WskbGyIsyh7IXfe9dQK2rtx-FlZgSoGOomt6_rd_yrkEea6NUQX1V5ugNWgLV7HhhpiY_4smLml3nBoVeFe7_rzxlMO8Ozw/s1600/TheAgeOfThePlatform.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4cHt9nBTC02kLbkpc6fxsJIqtBr1dZZy_GS1WYQuQ9Me0WskbGyIsyh7IXfe9dQK2rtx-FlZgSoGOomt6_rd_yrkEea6NUQX1V5ugNWgLV7HhhpiY_4smLml3nBoVeFe7_rzxlMO8Ozw/s1600/TheAgeOfThePlatform.jpg" width="130" /></a></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Please note that the word “platform” is usually ambiguous: it may mean a cloud/service platform (a back-end platform that serves a front-end app) or an open back-end that collaborates with a many apps (or other service platforms) through APIs. Here I use platform in the second sense; the first is always included in the “app” perimeter because of “<a href="http://searchconsumerization.techtarget.com/definition/device-agnostic-device-agnosticism">device agnosticism</a>”. That is, to let the end user pick whichever device is more suited to her current context, each mobile app must have a dual cloud service platform. So, in this post, each “app” comes with associated set of back-office/cloud services and I reserve “platform” for the implicit open innovation approach (cf. “<a href="http://organisationarchitecture.blogspot.fr/2014/03/entreprises-numeriques-et-homeostasie.html">L’age de la multitude</a>”)<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><h3 style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">2. Critical Mass and Software Usage </span></span></h3><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjp4ZR_bnCYHkcdL6IxYSC6ry-52GGiQS8x87NU1OILyKoG8tE4X1NzdD2VcbO-zIbJEWKzGnPXwxLvViHPCovQiVvdgRgiLVOXoZaadDmdObm1aShg7Bowxi2Hpu2dOr4LG1cZ3gyvQr0/s1600/Amas_de_percolation.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="226" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjp4ZR_bnCYHkcdL6IxYSC6ry-52GGiQS8x87NU1OILyKoG8tE4X1NzdD2VcbO-zIbJEWKzGnPXwxLvViHPCovQiVvdgRgiLVOXoZaadDmdObm1aShg7Bowxi2Hpu2dOr4LG1cZ3gyvQr0/s1600/Amas_de_percolation.png" width="320" /></a></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Even if software development is incremental, launching a successful product requires building a “critical mass” of value before one may start the “<a href="http://theleanstartup.com/">lean startup</a>” positive feedback loop of co-creation. This is the “V” in <a href="http://en.wikipedia.org/wiki/Minimum_viable_product">MVP</a> (Minimum Viable Product): there is a threshold of value brought to the user that one must pass before the <a href="http://en.wikipedia.org/wiki/Percolation_theory">percolation</a>model of viral adoption (helped with proper marketing) may kick in. The analogy with living organisms (“<i>SW as a living thing</i>”) is relevant here: software requires growth, constant change and a sustainable equation of user growth. The MVP aims to reach the tipping point when viral adoption becomes sustainable (what <a href="http://en.wikipedia.org/wiki/Eric_Ries">Eric Ries</a> calls “getting traction”).<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">The “critical mass of value” that is required for this tipping point varies considerably according to the pain point that the piece of software is trying to alleviate and according to the current state of the art. It may be measured in terms of function points (how rich an experience is necessary) and social weight. The complexity of modern experience comes from their social nature (if you think about it, most apps on your smartphone nowadays have a social component). Hence installing a new habit requires fighting against <a href="http://en.wikipedia.org/wiki/Metcalfe's_law">Metcalfe’s Law</a>, and <a href="http://organisationarchitecture.blogspot.fr/2012/06/ecologie-du-temps-libre-et-reseaux.html">displacing a previous social usage requires even more efforts</a>. The value critical mass may be large, which explains why some legacy Microsoft products such as <a href="http://en.wikipedia.org/wiki/Microsoft_Word">Word</a>, which I am still using to write technical papers, have not been easily replaced by open source alternatives.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">To reach this “value critical mass”, someone must invest an initial significant software development effort. In a complex world, where the risk to fail is high, one must reduce this initial software development cost, as we explained earlier. This is also a signature of the complex environment we live in: we must switch from ROI (return on investment) to the <a href="http://www.effectuation.org/learn/principle/3">affordable loss</a>principle.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">The percolation model shows why the ROI principle is no longer relevant: it is very hard to predict how well a successful MVP will percolate. The world is full of software startups with <a href="http://techcrunch.com/2014/02/19/facebook-buying-whatsapp-for-16b-in-cash-and-stock-plus-3b-in-rsus/">amazing valuations</a> because their app found its way to massive deployment and usage. But it would be very difficult to predict such success a few years earlier, when the MVP was still a prototype.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">The <b>affordable loss</b> approach means reducing the development cost to something that “you may afford to lose”. This means leveraging the previously shown layered architecture, and mostly leveraging the strength of the open source community.  <a href="http://en.wikipedia.org/wiki/Open-source_software">Open source software</a>is a machine that is constantly churning out software platforms that start their own journey towards fame and critical mass. The quality of open source software is directly related to its adoption (<b>because good software is built incrementally through feedback </b>– a key axiom). Adoption is inversely proportional to genericity, hence a using open source software is a “connected art”. One must understand the communities’ sizes and dynamics to select “the pieces of the puzzle”. Open source software yields by construction a nested / layered structure of software libraries with a combination of very high quality stable platforms for the common needs and more experimental gems with specific capabilities. This is why the word “<a href="http://scholarworks.umass.edu/ncdg/34/">ecosystem</a>” is so relevant to open source software. One should not think of a catalog of free software, but of a nested hypergraph of communities, where a collaborative price must be paid. This price is measured in (participation) time and (code) sharing. This represents a culture shift for most software organizations, but the efficiency of those who “play the game right” is such that it becomes <a href="http://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar">the only game worth playing</a>.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEia5FN0_B-3pNnYNwhYtIsD9m9VYjfRhuOpMPHEJvA4sE6gpZZkq0ga6LQn-mXCPtjEpuWkQiyPYai5dbwo_g9WWMEcxwcMUzAmPwjYj1iEkMzjyioMN1-7VPuBPNp24K6LMcnMY_AyzZs/s1600/LesG%C3%A9antsDuWeb.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEia5FN0_B-3pNnYNwhYtIsD9m9VYjfRhuOpMPHEJvA4sE6gpZZkq0ga6LQn-mXCPtjEpuWkQiyPYai5dbwo_g9WWMEcxwcMUzAmPwjYj1iEkMzjyioMN1-7VPuBPNp24K6LMcnMY_AyzZs/s1600/LesG%C3%A9antsDuWeb.jpg" /></a><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"> </span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This is just a hint of what a proper open source strategy should be, since there are so many aspects that I am not touching here. Open source is not only about software libraries, it is also about development methods, tools and processes, cloud computing and hardware, to name a few. As a follow-up, I would suggest reading Octo’s great book “<a href="http://www.geantsduweb.com/">Les Géants du Web</a>”, take a closer look at <a href="http://informationsystemsbiology.blogspot.fr/2011/11/lean-it-devops-and-cloud-programming.html">DevOps</a>or leverage the value that is found in the <a href="http://en.wikipedia.org/wiki/Open_Compute_Project">Open Compute</a> project.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><br /></span></div><h3 style="margin-bottom: 0.0001pt; text-align: justify;"><span style="font-family: Verdana, sans-serif;">3. The App developer's equation</span></h3><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><!--[if !supportLineBreakNewLine]--><!--[endif]--><o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><br /></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><br /></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This last part looks at app sustainability from the perspective of the developer. I have used the following (abstract) equation in my presentations for the past 10 years:<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><b><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Attractiveness = Market x Generosity x Value / Effort   <o:p></o:p></span></span></b></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">In this equation, <o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ul><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Market</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> is the size of the potential market size that a given platform is proposing. This equation was developed to understand the fate of mobile OS, but it applies to all kinds of platforms, from cloud service platforms that propose APIs (another dream of <a href="http://blog.programmableweb.com/2007/03/09/bt-launches-apis-for-telephony-mashups/">phone operators</a> for the past 10 years) to <a href="http://readwrite.com/2014/03/08/pebble-appstore-androids-apps#awesm=~oCE2EtJotB7pol">connected objects</a>.</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Generosity</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> is the share of the revenue (app price or advertizing revenue) that is sent back to the developer. For instance, Apple keeps a hefty 30%, whereas Android is more generous. Set-up costs should be factored in, for those platforms where some form of license or tools investment is still necessary.</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Value</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> is what the platform brings to the developer, as far as the end user experience is concerned. When I said “abstract” earlier, I meant that I don’t have a formula to measure “Value”. Most often, it is a judgment call from the developer, who evaluates what innovative and relevant services may be developed with the platform. There is subjectivity involved, such as the infamous “cool  factor”, which favors sets of APIs & features from which “cool stuff” may be built.</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Effort</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> is the amount of time it takes to build “one unit of value”. This is where the difference stands between great players, who provide the right <a href="http://en.wikipedia.org/wiki/Software_development_kit">SDK</a>, community support, testing services, and an efficient delivery (store) platform, and other less qualified players. Many APIs exposure programs have failed during the past 10 years because the effort expected from the developer was much too high. This is also why one should enroll help from qualified actors such as <a href="https://apigee.com/about/">Apigee</a> to develop an API strategy.</span></li></ul><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Roughly speaking, the equation is an abstract form of “Expected income / Expected Effort”. I have used this equation in the past years to explain why there would be two or at most three mobile open OS in the future, but it actually tells a lot of things.  You may understand why Microsoft announced (at last) that <a href="http://timesofindia.indiatimes.com/tech/tech-news/Microsoft-makes-Windows-free-for-small-devices/articleshow/33144292.cms">Windows Phone would be free in the future</a>. The “Market factor” yields a “winner take all”  dynamic that we have observed for many platforms (the <a href="http://en.wikipedia.org/wiki/Matthew_effect">Matthew effect</a> : the platform with the more users attracts more developers, benefiting from more open innovation, hence attracting more new customers). It also gives a few insights for a successful platform strategy:<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ol><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzuHxEe6o265N5PfFRf31PxYtMM0IeaD40mXXcWbSwCENqADmV-k-T7TExQjvovQ5WZ9p8p87phrQ6RFaHEE2IwZBLeYovJ8sEmDGr9CE1SBzkQsdVqzWjjRvcl7iwPqSlors1F2xdBCI/s1600/FunVsEffortHumor.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="312" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzuHxEe6o265N5PfFRf31PxYtMM0IeaD40mXXcWbSwCENqADmV-k-T7TExQjvovQ5WZ9p8p87phrQ6RFaHEE2IwZBLeYovJ8sEmDGr9CE1SBzkQsdVqzWjjRvcl7iwPqSlors1F2xdBCI/s1600/FunVsEffortHumor.jpg" width="320" /></a><li><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">Growth : get a critical mass of customers, as quickly as possible. To jumpstart the virtuous cycle (that is, enroll app developers while the market is still not here, one must use rewards and gamification – such as </span><a href="http://www.bbox-actus.com/voir_news.php?article=2639&filtre=all" style="font-family: Verdana, sans-serif; text-indent: -18pt;">hackathons</a><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">).</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Expose as much value from your APIs as possible, with a focus on differentiation that is exposing stuff which is both useful and not readily available elsewhere. This is probably the most strategic factor to predict the success of <a href="http://blog.hopkins.io/2013/07/25/withings-api/">connected objects</a> in the future, or larger domains such as the connected (smart) <a href="http://www.environmental-expert.com/articles/exclusive-interview-with-yves-caseau-executive-vice-president-at-bouygues-telecom-france-379180">home</a>.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Reduce the effort for the developer by embracing the open source and Web standards (languages, development tools, API styles, libraries, etc.). The adjacent illustration of the “Fun vs. Effort” is taken from a humorous site, but thinking in terms of <a href="http://organisationarchitecture.blogspot.fr/2012/06/ecologie-du-temps-libre-et-reseaux.html">value/effort is critical to system analysis</a>.</span></li></ol><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This equation gives also a way to evaluate the intrinsic value of a platform. It follows from what was said earlier that the value is the capacity to generate revenue streams from apps. This, in turn, is mostly related to accumulated user data. This leads to the idea, reported by <a href="http://en.wikipedia.org/wiki/Henri_Verdier">Henri Verdier</a>, that <a href="http://www.henriverdier.com/2011/02/mfg-labs-en-direct-de-la-strata.html">data is the new code</a>. The algorithms change constantly, and the best one are produced by external developers (hence the open innovation paradigm). What changes more slowly is the API structure and what accumulates over time is the amount of user data. This is important enough to be worth a future post when I report about the work of our group at the <a href="http://www.euro-case.org/index.php/academy/item/72-national-academy-of-technologies-of-france">NATF</a>on Big Data.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><!--[if !supportLineBreakNewLine]--><!--[endif]--><o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><br /><div class="MsoNormal" style="text-align: justify;"><br /></div></div>

Software Ecosystems and Application Sustainability

Today’s post is a set of simple, yet somehow deep, thoughts about the systemic nature of different ecosystems related to software. I was tra...

Read more »

Complex systems, Scale-free networks and Affiliation networks

<div dir="ltr" style="text-align: left;" trbidi="on"><div class="MsoNormal"><span style="font-family: Verdana, sans-serif;"><br /></span></div><div class="MsoListParagraph"><br /></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This post is a follow-up of the <a href="http://informationsystemsbiology.blogspot.fr/2013/12/seven-keys-for-complex-systems.html">previous one about complex systems</a>. I will further focus on the fifth point which said that efficiency in a complex system is strongly related to the capability to support information exchange flows, and will expand on the importance of <a href="http://en.wikipedia.org/wiki/Scale-free_network">Scale-free networks</a> . The importance and the size of information flows are directly related to the complexity of such systems, which is the amount of interaction between the components.<o:p></o:p></span></span></div><div class="MsoNormal"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">What makes a good information flow network for a complex system? Here are five characteristics that spring to mind: <o:p></o:p></span></span></div><div class="MsoNormal"></div><ul style="text-align: left;"><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Latency</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : to minimize the time it takes from some information to travel from one subsystem to the other.</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Throughput</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : to simultaneously transmit large amounts of information between all subsystems.</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Resilience</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">: to continue functioning when some subsystems or some links become unavailable.  These three characteristics are universal for all complex systems, for instance <a href="http://informationsystemsbiology.blogspot.fr/2007/02/five-challenges-for-entreprise.html">information systems</a> or <a href="http://organisationarchitecture.blogspot.fr/2008/02/mesurer-un-systme-runion.html">enterprise communication channels</a>.</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Searchability</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">: to ease the task of finding related subsystem through the exploration of the communication network. This property is related to dynamic growth and self-organization. In an <a href="http://informationsystemsbiology.blogspot.fr/2006/12/autonomic-computing.html">autonomic system</a>, automatic discovery of new features and new component is at the heart of the system’s dynamic organization.</span></li><li><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Cost</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">: to minimize the total weight of the communication network, whether we talk about energy, mass or dollars. Related to cost is the <b>scalability</b> of the communication network structure. Scalability means that the structure may easily evolve as the complex system grows.</span></li></ul><br /><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">To reduce latency, one must reduce the diameter of the network, which is (roughly) the average path length. The easier way is to add additional links. Similarly, to increase throughput and resilience, one must rely on path redundancy (the fact that many paths exist for routing one flow). However, there is a double trade-off: increasing the average degree and the number of edges both increases the cost and reduces <a href="http://books.google.fr/books?id=1gueFWR7qjoC&pg=PA154&lpg=PA154&dq=Kleinberg+condition+searchability&source=bl&ots=6vlHrNYrja&sig=u2plNAUPXA7UZUJYelfEgYh9vDU&hl=en&sa=X&ei=xDAGU_-dC-me0QWMwoGABg&redir_esc=y#v=onepage&q=Kleinberg%20condition%20searchability">searchability</a>. <o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Nature seems to have found the perfect solution for this trade-off with the <a href="http://en.wikipedia.org/wiki/Scale-free_network">scale-free network</a>structure. A scale-free network is a graph whose degree distribution follows a power law. Compared to a random graph, this means that there is a higher frequency of highly connected nodes, with large degrees. Scale-free have many wonderful properties, as explained by <a href="http://organisationarchitecture.blogspot.fr/2007/02/les-fondations-pour-ltude-des-rseaux.html">Duncan Watts</a> or <a href="http://en.wikipedia.org/wiki/Albert-L%C3%A1szl%C3%B3_Barab%C3%A1si">Albert-Laszlo Barabasi</a>. Their diameter is logarithmic in their size, and they are very resilient, that is their level of connectivity is weakly changed when some nodes become unavailable. The name “scale-free” comes from the self-similarity that the degree distribution implies. Somehow, a scale-free network may be seen as a “fractal structure”, which makes it an interesting candidate for self-growth and self-organization.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">What has been found in the past 20 years is that scale-free networks are everywhere, both in the nature-made complex systems (such as the network of chemical reactions within the brain), and in the human-grown systems that incorporate feedback and learning, such as the Web (network of pages) or the Internet (network of computers). Let me quote the introduction of « <a href="http://www.barabasilab.com/pubs/CCNR-ALB_Publications/200907-24_Science-Decade/200907-24_Science-Decade.pdf">Scale-Free Networks : A Decade and Beyond</a> » from Albert-Laszlo Barabasi : <i>“</i></span><i><span lang="EN-US">For decades, we tacitly assumed that the components of such complex systems as the cell, the society, or the Internet are randomly wired together. In the past decade, an avalanche of research has shown that many real networks, independent of their age, function, and scope, converge to similar architectures, a universality that allowed researchers from different disciplines to embrace network theory as a common paradigm.” </span></i><span lang="EN-US">As with any general big idea, this is an approximation of the real world, and there are some debates whether real networks have an exact power law for their degree distribution. Still, it is both a useful and powerful concept, when trying to design communication networks.<o:p></o:p></span></span></div><div class="MsoNormal"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US"><br /></span></span></div><div class="MsoNormal"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US"><br /></span></span></div><div class="MsoNormal"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><div class="separator" style="clear: both; text-align: center;"><span style="font-family: Verdana, sans-serif;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiq9Bsq80s0g2ruUl1E94PLIBcSOVAPwMixt-cmvJTgj8BaCkNYK-V2m8jQM8ndUHetNWeqtBcTTjyDJ9J1pMeecDvWW7yV5U3BZthl0AbcETCNY2sYIQVcfMu6QhwzPfDd9xYUIN3vaU0/s1600/LinkedBarabasi.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiq9Bsq80s0g2ruUl1E94PLIBcSOVAPwMixt-cmvJTgj8BaCkNYK-V2m8jQM8ndUHetNWeqtBcTTjyDJ9J1pMeecDvWW7yV5U3BZthl0AbcETCNY2sYIQVcfMu6QhwzPfDd9xYUIN3vaU0/s1600/LinkedBarabasi.jpg" /></a></span></div><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">I will now write a brief summary of « <a href="http://barabasilab.com/LinkedBook/">Linked</a> », a great book by Albert-Laszlo Barabasi. I have read this book many years ago, and <a href="http://organisationarchitecture.blogspot.fr/2009/11/systemes-humains-complexes-quelques.html">promised to give a review in my other blog</a>, but never found the time to do it. Still it is very relevant to what I just wrote (together with many other books which I have selected <a href="http://organisationarchitecture.blogspot.fr/2008/11/dix-ides-sur-la-structure-des-rseaux.html">in this post</a> ) since it contains a lot of details and examples about the importance of scale-free networks. The following is a short list of relevant key ideas that are well illustrated in this book, with no claim of completeness:</span></div><div style="text-align: justify;"><ul><li><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="text-align: left; text-indent: -18pt;">The book starts with the concepts of diameter and average path length. Throughout the book, many examples are given of really large networks with small diameters. For instance, the Web  (URL network) diameter is 19. Another interesting example is the molecule interaction network in a living cell, through chemical reactions. The “diameter” is only 3 (three degrees of separation). <a href="http://arxiv.org/abs/1111.4570">Lately we have learned</a> that the diameter of Facebook social graph is 4.7.</span></span></li><li><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="text-align: left; text-indent: -18pt;">By looking more closely at these networks, we see that the short diameter is not due to the number of edges but the presence of “connectors” (hubs), as defined by Malcom Gladwell in “<a href="http://en.wikipedia.org/wiki/The_Tipping_Point">The Tipping Point</a>” </span><span lang="EN-US" style="text-align: left; text-indent: -18pt;">J</span><span lang="EN-US" style="text-align: left; text-indent: -18pt;"> This is true for cell reactions, where a few molecules interact with many others. <a href="http://en.wikipedia.org/wiki/Small-world_network">Small-world networks</a>, as defined by Watts and Strogatz, also exhibit a higher <a href="http://en.wikipedia.org/wiki/Clustering_coefficient">clustering coefficient</a> than random graphs. These small-world structures may be thought of as small tightly connected groups, linked by connectors – hubs with high degrees.</span></span></li><li><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="text-align: left; text-indent: -18pt;">This leads to the concept of scale-free networks, by looking at the node degree distribution law. The presence of connectors is the result of <a href="http://en.wikipedia.org/wiki/Power_law">power laws</a>, which are also called “fat tailed” because the number of nodes with very high degree is much higher than a typical “<a href="http://en.wikipedia.org/wiki/Exponential_decay">exponential decay</a>” law. Another interesting example of scale-free networks is the graph of word co-occurrence in natural language.</span></span></li><li><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="text-align: left; text-indent: -18pt;">A good part of the book deals with how scale-free network may be grown, that is how they emerge in real life. This leads to the powerful “rich get richer” paradigm (also called <a href="http://en.wikipedia.org/wiki/Matthew_effect">the Matthew Effect</a>), where the probability of creating a new edge is proportional to the existing degree. Growth is a signature of Scale-free networks. I quote from the book : “<i>The power laws emerge – nature’s unmistakable sign that chaos is departing in favor of order. The theory of phase transitions told us loud and clear that the road from disorder to order is maintained by the powerful forces of self-organization and is paved by power laws</i>”.</span></span></li><li><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="text-align: left; text-indent: -18pt;">A very interesting part of the book deals with resilience, with examples drawn from biology such as the protein network in our metabolism. There is an interesting comparison with hierarchical networks (such as organizational charts in a traditional company or electricity distribution network) which are less fault-tolerant than scale-free networks (even with added redundancy for the high value links). Another quote: “<i>The coexistence of robustness and vulnerability plays a key role in understanding the behavior of most complex systems. Simulations have shown that the protein network refuses to break apart under randomly generated mutations</i>.”</span></span></li><li><span style="font-family: Verdana, sans-serif;"><span style="text-align: left;">Scale-free networks are graphs, with edges between two nodes that only describe binary interactions. Most of real world complex systems use more complex “n-ary” interactions, which could be described with </span><a href="http://en.wikipedia.org/wiki/Hypergraph" style="text-align: left;">hypergraphs</a><span style="text-align: left;">, </span><a href="http://faculty.ucr.edu/~hanneman/nettext/C17_Two_mode.html" style="text-align: left;">two-mode networks</a><span style="text-align: left;"> or </span><a href="http://obssr.od.nih.gov/training_and_education/issh/2012/files/ISSH%20Affiliation%20V1.pdf" style="text-align: left;">affiliation networks</a><span style="text-align: left;">. For instance, the meetings between coworkers in a company or the chemical reaction networks are hypergraphs. A meeting is an hyper-edge since it binds many participants; a chemical reaction is also an hyper-edge in the molecule graph. It is easy to model an affiliation network with a regular bi-partite graph (just add a few nodes for the hyper-edges), so this is not a big technical difference, but </span><a href="http://research.google.com/pubs/pub35369.html" style="text-align: left;">more and more interest is given to affiliation networks</a><span style="text-align: left;"> since they are very common in the real world of complex systems.</span></span></li></ul></div><br /><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Five years ago I decided to see if Duncan Watts’s results would also apply to Affiliation Networks. I wrote a paper entitled “<a href="http://www.researchgate.net/publication/260255723_Efficiency_of_Meetings_as_a_Communication_Channel_A_Social_Network_Analysis">Efficiency of Meetings as a Communication Channel : A Social Network Analysis</a>” which I presented at the “<a href="http://www.econbiz.de/events/event/4th-conference-on-management-social-networks-management-social-networks-dynamics-and-impacts/10009249132">Management and Social Networks</a>” conference in Geneva (2012). The main findings may be described as follows:<o:p></o:p></span></span></div><div class="MsoNormal"></div><ul style="text-align: left;"><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">I have shown that the most efficient meeting network structure relies on small meetings that have a high frequency. There is no surprise here, since this is a tenet of agile companies which are organized around daily short team meetings. Still, it is interesting to see that this is a deep structural property of the underlying network.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">I have proposed a “latency performance indicator” that predicts the speed of information propagation as  “ #of-monthly-meetings * log(#people-that-one-wants-to-communicate with) / log(#people-that-one-actually-meets-in-a-month)”. For those mathematically inclined, one may retrieve the best practices (<i>fewer meeting, frequent meetings, a few large meetings</i>) within the formula.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">The most interesting piece is the emergence of a <b>small-world structure</b> as the most efficient meeting network, which is a hybrid combination of small team meetings and a few larger meetings.  This reproduces, in the case of an affiliation network, the results found ten years ago by <a href="http://www.theguardian.com/books/2003/oct/18/featuresreviews.guardianreview11">Duncan Watts</a>. It tells that companies should reproduce the diversity found in nature, implement path redundancy and combine many really small and frequent meetings such as SCRUM <a href="http://en.wikipedia.org/wiki/Stand-up_meeting">stand-up meetings</a> together with a few overlapping “town-hall” meetings (large audiences).</span></li></ul><br /><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Scale-free networks are similar to what sociology calls “<a href="http://en.wikipedia.org/wiki/Ambidextrous_organization">ambidextrous organizations</a>”.  Ambidextrous organizations leverage the power of cliques and the strength of weak ties. The “power of cliques” is precisely the strength of team work, small group of people that are all connected to one another (hence the <a href="http://en.wikipedia.org/wiki/Clique">clique</a>name), establishing “strong ties” (which means frequent in the world of social network science). The “<a href="http://acawiki.org/The_strength_of_weak_ties">strength of weak ties</a>” is the law established by Mark Granovetter that says that we need to use our “weak ties”/extended network to get out from difficult or exceptional situations. “Weak ties” refer to people that we see rarely (as opposed to strong ties) ; the “weak ties” make the edge of our social network, they provide the diversity of viewpoint and culture which is often absent from the core of our social network (since “strong ties” tend to be very similar to ourselves).<o:p></o:p></span></span></div><div class="MsoNormal"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><div class="MsoNormal"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjb1iTR_HSgPujEKH9aAqLns-n8-JwCbvW0ZVf9E7XtH6MagCDtg957prnnyWDOHjHk_fIayMAie46Ar1-7CtxSjrTMsMLssBU5KVmFKWwY_juObzlhKqGVmPg9pQq5xB3G8-Vf56SPkJE/s1600/Holacracy.Double_Linking.JPG" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="185" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjb1iTR_HSgPujEKH9aAqLns-n8-JwCbvW0ZVf9E7XtH6MagCDtg957prnnyWDOHjHk_fIayMAie46Ar1-7CtxSjrTMsMLssBU5KVmFKWwY_juObzlhKqGVmPg9pQq5xB3G8-Vf56SPkJE/s1600/Holacracy.Double_Linking.JPG" width="320" /></a></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">The idea that complex systems sciences in general and social network structures in particular, are relevant to enterprise organization is becoming more and more popular (this is precisely the topic of my other <a href="http://organisationarchitecture.blogspot.fr/2012/12/organisations-excellence-et-management.html">blog</a>).  I will conclude with three examples which are closely related to this post, since those three theories attempt to improve management efficiency through a better-designed information network:<o:p></o:p></span></span></div><div class="MsoNormal"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><div class="MsoNormal"></div><ul style="text-align: left;"><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><a href="http://en.wikipedia.org/wiki/Sociocracy">Sociocracy </a> uses circles as a team structure, and <a href="http://en.wikipedia.org/wiki/Double_linking">doublelinks</a> (each intersection between circles is represented by two individuals) to implement redundant information propagation paths. The illustration is taken from <a href="http://en.wikipedia.org/wiki/File:Holacracy.Double_Linking.JPG">Wikipedia</a>.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><a href="http://www.betacodex.org/">BetaCodex</a> is a management theory and practice whose claim is to “organize for complexity”. It is based on a cellular network structure, which draws its organizing principles from biology. The tree structure is replaced by a denser network of circles (with a clear reference to sociocracy), providing shorter and more resilient information propagation paths.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><a href="http://en.wikipedia.org/wiki/Holacracy">Holacracy</a> is another recent management theory that draws on <a href="http://informationsystemsbiology.blogspot.fr/2013/12/seven-keys-for-complex-systems.html">complex system theory</a>. Here again we find a system of self-organizing circles (with a similar influence from sociocracy). The most defining feature of holacracy is “to organize around purpose” (cf. the fourth principle of our previous list).</span></li></ul><br /> <br /><div class="MsoNormal"><br /></div></div>

Complex systems, Scale-free networks and Affiliation networks

This post is a follow-up of the previous one about complex systems . I will further focus on the fifth point which said that efficiency in a...

Read more »

Seven Keys for Complex Systems Engineering

<div dir="ltr" style="text-align: left;" trbidi="on"><span style="font-family: Verdana, sans-serif;"><br /></span><br /><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">I gave a <a href="http://www.dailymotion.com/video/xxvknb_intervention-yves-caseau-inauguration-irt-systemx_tech">talk</a>early this year at the “<a href="http://www.irt-systemx.fr/">IRT SystemX</a>” inauguration, about the challenges that occur when engineering “<a href="http://en.wikipedia.org/wiki/System_of_systems">Systems of Systems</a>”. This talk is a quick introduction of what we can learn from <a href="http://en.wikipedia.org/wiki/Complex_systems">complex systems</a> when designing large-scale interactive industrial systems. Complex systems are defined by their goals (purpose) and a set of sub-systems with rich interactions. The complexity of these interactions yields the concept of <a href="http://en.wikipedia.org/wiki/Emergence">emergent</a> behavior. Complex systems have a <a href="http://en.wikipedia.org/wiki/Fractal">fractal</a>nature, that is, they exhibit multiple scales, both from a physical/descriptive level and from a temporal level. Complex systems embed memory and have the capability to learn, which makes them both dynamic and adaptive systems. They interact constantly with their environment, which means that a dynamic vision of flows is more relevant than a static description of their top-down decomposition. Most complex systems renew their low-level components in a continuous process. <a href="http://en.wikipedia.org/wiki/Teleonomy">Teleonomy</a>and process analysis are, therefore, the most useful approach to capture the essence of a complex system.</span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">I have become gradually fascinated by the topic of complex systems because I find it everywhere in my job and my own research<i>. Complex systems</i> is the right framework to understand the management and the organization of modern enterprises. This is the topic of my other <a href="http://organisationarchitecture.blogspot.fr/2008/10/entreprise-et-systmes-complexes.html">blog</a>. All that is said about complex systems in the previous paragraph applies to a company. I also found that this applies to <a href="http://www.amazon.com/Information-Technology-Chief-Executive-Organization-ebook/dp/B006HC9OXY/ref=sr_1_1?ie=UTF8&qid=1388473942&sr=8-1&keywords=Yves+CASEAU">information systems</a> as well. The main reason for creating this <a href="http://informationsystemsbiology.blogspot.fr/2006/11/welcome-message.html">blog</a>was the realization that the proper control for information system has to be emergent, following the lead of <a href="http://en.wikipedia.org/wiki/Kevin_Kelly_(editor)">Kevin Kelly</a> and the intuition behind <a href="http://informationsystemsbiology.blogspot.fr/2006/12/autonomic-computing.html">Autonomic Computing</a>. Last, complex systems are everywhere when one tries to understand the most common business ecosystems, such as smartphone application development, <a href="http://www.environmental-expert.com/articles/exclusive-interview-with-yves-caseau-executive-vice-president-at-bouygues-telecom-france-379180">smart homes</a> or smart grids. I have talked about <a href="http://informationsystemsbiology.blogspot.fr/2013/05/systemic-simulation-of-smart-grids-s3g.html">Smart Grids Players as a Complex System</a> in this blog. More examples may be found in <a href="https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&ved=0CDQQFjAB&url=http%3A%2F%2Flink.springer.com%2Fcontent%2Fpdf%2F10.1007%252F978-3-642-34404-6_2.pdf&ei=z3DCUujtPIa40QW8q4CYBw&usg=AFQjCNFNTyp0qeVGD3ncRTnmofUk4wVzyw&sig2=HXo3VD1WKNA">my keynote at CSDM 2012</a>.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">There is a <b>paradox</b> with the popularity of “complex systems science” in today’s business culture. On the one hand, the importance of complex systems’ concepts is obvious everywhere: system of systems, enterprises, markets. On the other hand, the practical insights are not so clear. « System thinking » has become a buzzword and the word “complexity” is everywhere … still many textbooks and articles which claim to apply “the latest of complex science theory” to business and management problems are either obscure or shallow. This is not to say that there does not exist a wealth of knowledge and practical insights that is available in complex systems literature. On the contrary, the following is a selection of some of the books which I have found useful during the last few years.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivLdDh4Ze6uTg6RBnpzPelsoQvL4fD5xrB98Kvs08bCIjRUe_URcTG1RceJCXlmKhvHO-qkXTwKAS6zA8WqfXQfQA0SvCK0nW90SVJEj2heBODj833PzAfZhZbkH5-5t6kkWm-FZMYmwg/s1600/BiblioSystemesSlide.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivLdDh4Ze6uTg6RBnpzPelsoQvL4fD5xrB98Kvs08bCIjRUe_URcTG1RceJCXlmKhvHO-qkXTwKAS6zA8WqfXQfQA0SvCK0nW90SVJEj2heBODj833PzAfZhZbkH5-5t6kkWm-FZMYmwg/s640/BiblioSystemesSlide.jpg" width="640" /></a></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">Today’s post is a crude and preliminary attempt to pick <b>seven keys</b> that I have found in these books which, to me at least, are practical in the sense that they unlock some of the complexity – or mystery – of the practical complex systems which I have encountered. </span>There is no claim of completeness or rigorous selection. <span lang="EN-US">This is clearly a personal and subjective list which I consider a « work in progress ». This is just a list, so I will not develop each of the seven keys here, although each would deserve a blog post of  its own.<o:p></o:p></span></span></div><div class="MsoListParagraphCxSpFirst" style="margin-bottom: 0.0001pt;"><br /></div><div class="MsoListParagraphCxSpFirst" style="margin-bottom: 0.0001pt;"></div><ol style="text-align: left;"><li><div style="text-align: justify;"><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Complexity means that forecasting is at best extremely slippery and difficult, and most often outright impossible</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">. This is, for instance, the key lesson from Nassim Taleb’s books, such as <a href="http://organisationarchitecture.blogspot.fr/2009/05/simulation-jeux-et-previsions-face.html">The Black Swan</a>. The non-linearity of complex system interactions causes the famed <a href="http://en.wikipedia.org/wiki/Butterfly_effect">butterfly effect</a>, in all kinds of disciplines. If you line up a series of queues, such as in the <a href="http://en.wikipedia.org/wiki/Beer_distribution_game">Beer Game</a> supply chain example, each queue amplifies the variations produced by the previous one and the result is very hard to forecast, hence to control (this depends, obviously, of the system load). This does not mean that simulation of complex systems is useless, it means that is must be used for <i>training</i>as opposed to <i>forecasting</i>. Following <a href="http://suntzusaid.com/">Sun Tzu</a> or <a href="http://organisationarchitecture.blogspot.fr/2009/05/simulation-jeux-et-previsions-face.html">François Jullien</a>, one must practice “serious games” (such as war games) to learn about complex system from experience. This complexity also means that one needs as much data as possible to understand what is happening, and should beware of simplified/abstract description. “<a href="http://en.wikipedia.org/wiki/God_is_in_the_detail">God is in the detail</a>” has become a very popular business idiom in the last decades.</span></div><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></li><li><div style="text-align: justify;"><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Complex systems most often live in a complex environment which makes homeostasis an (increasingly) complex feast of change management</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">. <a href="http://en.wikipedia.org/wiki/Homeostasis">Homeostasis</a> describes the process through which a complex system continuously adapts to its changing environment. The characteristic of successful complex systems, in a business context, is the ability to react quickly, with a large range of possible reactions. This applies both at the level of what the system does and what it is capable of doing. This is illustrated by the rise of the word “agility” in the business vocabulary. The <a href="http://en.wikipedia.org/wiki/Variety_(cybernetics)">law of requisite variety</a> tells us why detailed perception is crucial for a complex system (which is clearly exemplified by recent <a href="http://www.youtube.com/watch?v=iC_WKbAR8IU">robots</a>) : the system’s representation of the environment should be as detailed/varied as the sub-space from the outside environment that the homeostatis process needs to react to.</span></div><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></li><li><div style="text-align: justify;"><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Complex systems, because of the non-linear interaction in general, and because its components have both memory and the capability to learn, exhibit statistical behaviors which are quite different from “classical” (Gaussian) distribution</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">. This is one of the most <a href="http://organisationarchitecture.blogspot.fr/2009/11/systemes-humains-complexes-quelques.html">fascinating insights</a> from complex systems theory: fat tails (<a href="http://en.wikipedia.org/wiki/Power_law">power laws</a>) are the signature of intelligent behavior (such as learning).   In classical physics or statistics, all individual events are (most often) assumed to be independent, which yields <a href="http://en.wikipedia.org/wiki/Law_of_large_numbers">the law of large numbers</a>and <a href="http://en.wikipedia.org/wiki/Normal_distribution">Gaussian distributions</a>. But when the individual events are caused by actors who can learn or influence each other, this is no longer true. Rather than the obvious reference to Nassim Taleb, the best book I have read on this is <a href="http://www.nytimes.com/2013/01/06/books/review/the-physics-of-wall-street-by-james-owen-weatherall.html?_r=0">The Physics of Wall Street</a>. This works both ways: it warns us that “<a href="http://en.wikipedia.org/wiki/Black_swan_theory">black swans</a>” should be expected from complex systems, but also tells us that some form of coordinated behavior is probably at work when we observe a <a href="http://en.wikipedia.org/wiki/Fat-tailed_distribution">fat tail</a>. There is another interesting consequence : <i>small may be beautiful</i> with complex systems, if adding many similar sub-systems creates un-foreseen complexity ! Classical statistics is all in favor of large scale and centralization (reduction of variability) whereas complex behavior may be better understood with a de-centralized approach. This is precisely one of the most interesting debates about the <a href="http://informationsystemsbiology.blogspot.fr/2011/09/systemic-simulation-of-smart-grids.html">smart grids</a> : if there is no feedback, learning and user behavior change, the linear nature of electricity consumption favors centralization (and large networks); if the opposite is true, a system of system approach may be the best one.</span></div><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></li><li><div style="text-align: justify;"><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Resilience in complex systems often comes from the distribution of the whole system purpose to each of its subcomponents</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">. This is another great insight from complex system theory: control needs to be not only distributed (to sub-systems) but also declarative, that is, the system’s purpose is distributed and the control (deriving the action from the purpose) is done “locally” (at the sub-system level). This idea of embedding the whole system’s purpose into each component is often referred as the <a href="http://www.intechopen.com/download/get/type/pdfs/id/13844">holographic principle</a>, with a nice hologram metaphor (in each piece of a <a href="http://en.wikipedia.org/wiki/Holography">hologram</a>, there is a “picture” of the whole object). This principle has been proven many times experimentally with information systems’ design: it has produced “policy-based control”, where the goals/SLA/purposes are distributed in a declarative form (hence the word “policy”) to all sub-components. I gave the example of <a href="https://www.slapos.org/">SlapOS</a>in my IRT talk as a great illustration of this principle. This is also closely related to the need for fast reaction in the homeostasis process: agility requires distribution of control, with a bottom-up / networked organization similar to living organisms (for most critical functions). One of my favorite books which apply this to the world of enterprise organization is “<a href="http://innovationwatch.com/managing-the-evolving-corporation-by-langdon-morris-van-nostrand-reinhold/">Managing the Evolving Corporation</a>” by Langdon Morris.</span></div><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></li><li><div style="text-align: justify;"><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Efficiency in a complex system is strongly related to the capability to support information exchange flows</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">. There is a wealth of information about <a href="http://barabasilab.com/LinkedBook/">the structure of information networks</a>that best support these flows. <a href="http://en.wikipedia.org/wiki/Scale-free_network">Scale-free networks</a>, for instance, occur in many complex systems, ranging from the Web to the molecular interactions in living cells and including <a href="http://organisationarchitecture.blogspot.fr/2008/11/dix-ides-sur-la-structure-des-rseaux.html">social networks</a>. Scale-free networks reduce the average diameter, among other interesting properties, and can be linked to avoiding long paths in communication chains, both for agility and resilience. The challenge that these information flows produce is represented by the product of the interaction richness (essence of complexity in a complex system) and the high frequency of these interaction (our key #2) – the product of two large numbers being an even larger number.  My other <a href="http://organisationarchitecture.blogspot.fr/2009/08/business-process-communication-model.html">blog</a>is dedicated to the idea that managing the information flows is the most critical management challenge for the 21^st century (an idea borrowed from “<a href="http://www.amazon.com/Organizations-James-G-March/dp/063118631X">Organizations</a>” by March & Simon). For instance, the necessity to avoid long paths translates into versatility : complexity prevents specialization, because too much specialization generates even more synchronization flows. This communication challenge is not simply about capabilities (“the size of the communication pipes”), it is also about semantics and meaning. A common vocabulary is essential to most “systems of systems”, whether they are industrial systems or companies.</span></div><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></li><li style="text-align: justify;"><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Complexity in time is something that is difficult to appreciate for humans</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">. One of the most critical aspect of complex systems are the loops, mostly feedback loops. Peter Senge and <a href="http://web.mit.edu/jsterman/www/BusDyn2.html">John Sterman</a> have written <a href="http://en.wikipedia.org/wiki/The_Fifth_Discipline">famous books</a> about this.  Reinforcement and stabilizing loops are what matter the most when trying to describe a complex system, precisely because of their non-linear natures. The combination of loops, memory and delays cause surprises to human observers. John Sterman gives many examples of overshooting, which happen when human over-react because of the delay. Kevin Kelly gives similar examples related to the management of wildlife ecosystem. The lesson from nature is a lesson of humility : we are not good at understanding delays and their systemic effects in a loop. In the world of business, we have a difficulty to understand long-term consequences of our actions, or simply to visualize long-term equilibriums. Many people think that user market share and sales market share should converge, given enough years, without seeing the bigger picture and the influence of attrition rate (churn). Even simple laws such a <a href="http://en.wikipedia.org/wiki/Little's_law">Little’s Law</a> may produce counter-intuitive behaviors.<br /></span></li><li style="text-align: justify;"><b style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"><span lang="EN-US">Efficient control for complex systems is an emergent property</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;">. Control strategies must be grown and learned, in a bottom-up approach as opposed to a top-down design. We are back to <a href="http://informationsystemsbiology.blogspot.fr/2006/12/autonomic-computing.html">autonomous computing</a> : top-down or centralized control does not work. It may be seen as another consequence of Ross Ashby law of requisite variety: complete control is simply impossible. Adaptive control required autonomy and leaning. This is, according to me, the key insight from Kevin Kelly’s book, <a href="http://kk.org/outofcontrol/">Out of Control</a> : “</span><span lang="EN-US" style="background-color: #f6f6f6; font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;">«<span class="apple-converted-space"> </span></span><em style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;">Investing machines with the ability </em><em style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"><span lang="EN-US">to adapt on their own, to evolve in their own directions, and grow without human oversight is the next great advance in technology. Giving machines freedom is the only way we can have intelligent control</span></em><span lang="EN-US" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"> ».  This insight is closely related to our key #4 : autonomy and learning transform progressively distributed policies into emergent control.  There exists another corollary from this principle: such policies, or rules, should be simple, and <a href="http://organisationarchitecture.blogspot.fr/2012/09/regles-simples-et-situations-complexes.html">the more complex the system, the simpler the rules</a>. One could say that this is nothing more than the old idiom <a href="http://en.wikipedia.org/wiki/KISS_principle">KISS</a>, a battlefield lesson from engineering lore. But <a href="http://vimeo.com/47706484">there is more to it</a>, there seems to be a systemic law that is comforted by business experience: only simple explicit rules provide long-term values to complex systems. Any rule that is complex has to be implicit, that is constantly challenged and re-learned.</span></li></ol><br /><div class="MsoListParagraphCxSpLast" style="margin-bottom: 0.0001pt;"><br /></div></div>

Seven Keys for Complex Systems Engineering

I gave a talk early this year at the “ IRT SystemX ” inauguration, about the challenges that occur when engineering “ Systems of Systems ”. ...

Read more »

Lean Startup & Lean Innovation Factory

<div dir="ltr" style="text-align: left;" trbidi="on"><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="text-align: left;"><br /></span></span><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="text-align: left;">I had the privilege to attend the <a href="http://www.lean-it-summit.com/">Lean IT Summit</a>in Paris a week ago, and was pleased to hear “<a href="http://theleanstartup.com/">The Lean Startup</a>” mentioned in almost half of the talks. Actually, the <i>Lean Startup</i> is so popular that some are <a href="http://venturebeat.com/2013/10/16/lean-startups-boo/">getting annoyed</a></span><span lang="EN-US" style="text-align: left;">:)</span><span lang="EN-US" style="text-align: left;"> I co-wrote the preface of the French edition because <a href="http://organisationarchitecture.blogspot.fr/2012/03/linnovation-digitale-est-dans-le-code.html">I am a strong believer in the principles that Eric Ries explains in his book</a>. However, with popularity comes exaggeration and re-interpretation. Here are two things I heard during the lean IT summit that got me annoyed as well:</span></span></div><div class="MsoNormal" style="text-align: center;"></div><ul><li style="text-align: justify;"><i style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"><span lang="EN-US">The Lean Startup</span></i><span lang="EN-US" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"> is what the lean community has expressed for a long time, with better words. Kudos to Eric Ries for being such a great communicator !</span></li><li style="text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><i>The Lean Startup</i> is a lean reformulation of well-known innovation practices. Actually, <a href="http://www.lean.org/LeanPost/Posting.cfm?LeanPostId=84#.UmPf7yhdV8G">innovation is in the genes of lean manufacturing</a>, so no surprise there !</span></li></ul><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">I disagree on both accounts:</span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">The Lean Startup is not a book about lean, it’s a book about innovation, mostly startups but which is also relevant for larger companies, which is why I am such a strong advocate. After writing part of the preface, I ordered many dozens of the book which I have distributed freely in my own company. Sure, the lean framework gives a lot of sense to the overall contribution, but this is not the point.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Although many of the key ideas have been around for a while, the combination of these principles into a well-defined innovation process is a true contribution. It definitely goes against what most people believed to be innovation in larger companies. I had heard Eric Ries’s ideas expressed by a few VC from Silicon Valley, but they were anything but mainstream.</span></li></ul><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Hence this short post is about two things. The first part is a “<i>Lean Startup for dummies</i>” summary. It is by no means thorough nor complete, <a href="http://organisationarchitecture.blogspot.fr/2011/08/lean-startup-et-lean-product.html">my French post from two years ago did a better job</a>, but it is written for the corporate world and emphasizes what may be seen as “different”, at least compared with how “innovation” was described ten years ago, when we talked about “ideation factories”. The second part describes what I call “<b>Lean Innovation Factory</b>”, that is the application of Lean Startup principles to the innovation division of a large company.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><br /></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b><span lang="EN-US"><span style="font-size: large;">1. Lean Startup for dummies</span></span></b><span lang="EN-US" style="font-size: 12pt;"><o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><br /></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><a href="http://theleanstartup.com/">Eric Ries’s book</a> deserves to be read because it is filled with meaningful examples. Therefore, a short summary cannot do justice to its content. Here I will only pick three key principles:<o:p></o:p></span></span><br /><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">(a)<span style="font-size: 7pt; font-weight: normal;">    </span></span></b><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Innovation is about doing, not about producing ideas</span></b></div><div class="MsoListParagraphCxSpMiddle" style="text-align: justify;"><br /></div><div class="MsoListParagraphCxSpMiddle" style="margin-left: 0cm; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">This principle is very similar to what the</span><span lang="EN-US"> </span><a href="http://www.pretotyping.org/the-pretotyping-manifesto-1"><span lang="EN-US">pretotyping  manifesto </span></a><span lang="EN-US">promotes. The prototyping manifesto gave us these mottos:  <i> innovators beat ideas, pretotypes beat productypes, building beat talking… </i>which all tell that the key part in innovation is the doing. This is especially true in the digital world, and is acknowledged by similar mottos from Google (“Focus on the user and all else will follow”, “Fast is better than slow”) or Facebook (“code wins”, ”done is better than perfect”). To innovate means, most of the times and above everything else, to meet a customer problem and to remove a pain point. Value creation occurs at the contact with the customer, not in a brainstorming room. This does not mean that ideation tools and techniques are not useful or important; it means that only “<i>on the gemba</i>” can we check that innovation actually works. </span></span></div><div class="MsoListParagraphCxSpMiddle" style="margin-left: 0cm; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US"><br /></span></span></div><div class="MsoListParagraphCxSpMiddle" style="margin-left: 0cm; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">This is more revolutionary than it may sound for larger and older companies, which have associated the “innovation” word with “great ideas”. I have in my library dozens of book about innovation that distinguish between all kinds of innovation (according to the source of the “newness”) and that propose many processes for reaching all kinds of customers. The beauty of the <i>lean startup </i>framework is to simplify – so to speak, since value-creation-at-the-hands-of-the-customer is indeed hard – and to get rid of all the innovation funnels and ideation laboratory paraphernalia. What is clear to me after 15 years in the world of telecommunication service innovation is that everyone has the same ideas, the difference between success, failure and doing nothing (the most frequent case) is the quality of the execution process.<o:p></o:p></span></span><br /><span style="font-family: Verdana, sans-serif;"><span lang="EN-US"><br /></span></span><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">(b)<span style="font-size: 7pt;">   </span></span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><b>Innovation requires iteration since nobody gets it right the first time.</b></span></div><div class="MsoListParagraphCxSpMiddle" style="margin-left: 0cm; text-align: justify;"><br /></div><div class="MsoListParagraphCxSpLast" style="margin-left: 0cm; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">This principle is often associated with the motto:  </span><i><a href="http://www.ssireview.org/blog/entry/fail_faster_succeed_sooner/"><span lang="EN-US">fail fast to succeed sooner</span><span lang="EN-US" style="font-style: normal; mso-ansi-language: EN-US;"> </span></a></i><span lang="EN-US">.  In the Lean Startup world, it leads to the MVP: <a href="http://en.wikipedia.org/wiki/Minimum_viable_product">minimum viable product</a>. Each word is important: a MVP is a product that may be placed in our customers’hands (this is not a prototype, it may be simple but it should not be fragile). A MVP is “viable” when it solves the customer’s problem. Its role is to jumpstart an iterative process of feedback collection, which may only happen if the customer finds a practical interest with the MVP, on the first day. A MVP is “minimal” because it is “<i>as simple as possible but not simpler</i>”, to paraphrase Einstein. This allows us to start the iteration as soon as possible, but not sooner. This emphasis on iteration echoes what a venture capitalist from Silicon Valley told me six years ago: there is no correlation between the success of a software startup and the quality of the piece of code that is shown to the early investors. On the other hand, there is a clear correlation between success and the ability to listen to the feedback of early customers and turn them into improvements. </span></span></div><div class="MsoListParagraphCxSpLast" style="margin-left: 0cm; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US"><br /></span></span></div><div class="MsoListParagraphCxSpLast" style="margin-left: 0cm; text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">This is also a bigger difference than one may think with the prevailing culture of large companies. It goes against the myth “<i>you must get it right on the first time; you have only one chance to make the right impression</i>”. The common culture of detailed market studies, coupled with the practice of lengthy marketing requirements, is replaced by a “hands-on” culture. MVP is a process that co-constructs software code, requirement and detailed specifications at the same time.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><br /><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">(c)<span style="font-size: 9px;"> </span></span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><b>A successful business model is built iteratively using customers’ feedbacks.</b></span></div><div class="MsoListParagraph" style="text-align: justify; text-indent: -18pt;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US"><br /></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">A successful business model is not a <i>pre-condition</i> but a <i>post-condition</i> for the innovation process. A startup is a “business model factory”; this is well understood today by the various startup “incubators” and “accelerators”  and it may be acknowledged as one of Eric Ries’ contributions. To make a “business model factory” deliver, one needs three things. First, we need to set up <b>measurement points</b> in our MVP. We need to measure usage and value creation, that is, how the problem is being solved. Second, we need to build and then validate a <b>value creation model</b>, which Eric Ries calls</span><span lang="EN-US"> </span><a href="http://lexicon.ft.com/Term?term=innovation-accounting"><i><span lang="EN-US">innovation accounting</span></i></a><span lang="EN-US">. This is the direct application of the old saying “a measure is worth nothing without a model” (without a model, one does not known how to interpret a measure). This is an iterative process and not an exact science, where trials and errors is the common approach. On formulate hypotheses, which are either validated on invalidated by the collected measurements. Eric Ries is adamant in his book about preferring facts to opinions </span><span lang="EN-US">:)</span><span lang="EN-US">. Last, when the model fails, the startup needs to “<b>pivot</b>”, that is to formulate a new value creation hypothesis. A key contribution from The Lean Startup is the wealth of examples and explanations regarding business models and pivoting. </span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US"><br /></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">This third principle is no less of a rupture with respect to the sanctity of the business case and its return on investment (RoI) that is observed in many large companies. It is simply not possible to formulate a credible business case when one starts to innovate. Obviously, one needs to start somewhere, hence there must be some initial hypotheses regarding value creation. However, the business model for the MVP is the result of an iterative process; the good news is that it comes with the validation provided by usage measures.</span><span lang="EN-US"><o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><br /></div><div class="MsoNormal" style="text-align: justify;"><br /></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b><span style="font-size: large;">2. Lean Innovation Factory</span></b><o:p></o:p></span></div><div class="MsoNormal"></div><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><br /></span></div><span lang="EN-US"></span><br /><div style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">I have started to use the term « Lean Innovation Factory » as a way to encapsulate principles from The Lean Startup applied to the innovation division of a large company, such as the one that I manage at Bouygues Telecom.</span><span style="font-family: Verdana, sans-serif;">  </span><span style="font-family: Verdana, sans-serif;">The name </span><b style="font-family: Verdana, sans-serif;">Lean Innovation Factory</b><span style="font-family: Verdana, sans-serif;"> (LIF) captures three ambitions:</span></span></div><span lang="EN-US"></span><div style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;"><br /></span></span></div><span lang="EN-US"><div style="text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">(1)<span style="font-size: 7pt;">    </span></span><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">It is an innovation factory</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">.</span></div><div style="text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></div><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">A “lean innovation factory” is a process that produces innovations. An innovation is a product or service that solves a problem, which is demonstrated </span><i style="font-family: Verdana, sans-serif; text-indent: -18pt;">in the hands of a customer</i><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">. The process does not need to deliver a full-scale solution to prove its effectiveness, it can operate on a smaller set of customers, but only the “monitored feedback” of real users will validate the creation of innovative value. The emphasis is on “doing” and “building”; ideas have no glorified status in the Lean Innovation Factory, we strive for physical products and running software. We make ours the words of </span><a href="http://en.wikipedia.org/wiki/W._Edwards_Deming" style="font-family: Verdana, sans-serif; text-indent: -18pt;">W. Edwards Deming</a><span style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : “<i>In God we trust, all others must brings data</i>”.</span></div><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></div><div style="text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">(2)<span style="font-size: 7pt;">   </span></span><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">It follows the “Lean Startup” principles</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">.</span></div><div style="text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></div><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">The engine for creating value is the iteration of MVP feedback, which means that we strive to build the first MVP as quicky</span><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">  </span><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">as possible (fail fast to succeed sooner), but while keeping the meaning of “viable” into our minds : the MVP is not a prototype, it is a product. We implement the heart of innovation accounting, in the sense that we measure feedback and we build continuously a value creation model that is validated or invalidated by our users.</span></div><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></div><div style="text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">(3)<span style="font-size: 7pt;">    </span></span><b style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">As a “factory”, the process is as important as the end result, because the result keeps changing while the strengths and the skills of the “factory workers” may build up</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">.</span></div><div style="text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><br /></span></div><div style="text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">This is the same pitch that I made for the “<a href="http://informationsystemsbiology.blogspot.fr/2012/07/lean-software-factory.html">lean software factory</a>”, and a reason for choosing a similar name </span><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">:) To build a lean innovation factory is not only to build great product or service innovations, it means to build an organization that learns to do this better and better over time. This is clearly what Eric Ries tries to teach from his own experience with many startups, and where the link with "<a href="http://en.wikipedia.org/wiki/The_Toyota_Way">The Toyota Way</a>" is the most evident.</span></div></span><br /><div class="MsoNormal" style="text-align: justify;"><br /></div><div class="MsoNormal" style="text-align: justify;"><br /></div><div class="MsoNormal" style="text-align: justify;"><br /></div><br /><div class="MsoNormal" style="text-align: justify;"><br /></div></div>

Lean Startup & Lean Innovation Factory

I had the privilege to attend the Lean IT Summit in Paris a week ago, and was pleased to hear “ The Lean Startup ” mentioned in almost half ...

Read more »

Follow-up on Lean Architecture

<div dir="ltr" style="text-align: left;" trbidi="on"><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This post is a sequel to the previous post <a href="http://informationsystemsbiology.blogspot.fr/2013/01/lean-architect-do-we-need-abstraction.html">regarding the lean architect</a>. It’s main topic is the book review of “<b>Lean Architecture</b> – <i>for Agile Software Development</i>”  from James O. Coplien & Gertrud </span><span lang="EN-US" style="font-family: Verdana, sans-serif;">Bj</span><span lang="EN-US" style="font-family: Verdana, sans-serif;">ø</span><span lang="EN-US" style="font-family: Verdana, sans-serif;">rnvig</span><span style="font-family: Verdana, sans-serif;">  This is really a follow-up in the sense that I have found most of the ideas from my previous post expressed in this book, but they are more thoughtfully presented :).  It also goes further than my previous analysis, which is why I am writing this quick – and incomplete – book review. James Coplien  is both a prolific author and a serious expert on software architecture, with an itinerary that is not so different from mine, especially with respect to object-oriented programming, which use to be my own domain of expertise twenty years ago. Similarly <a href="http://informationsystemsbiology.blogspot.fr/2009/04/selected-bibliography.html">the 10 years which I have spent working on information systems architecture</a>from 1997 to 2007, are well mirrored by Coplien track record in system’s architecture in the 90s and 00s.</span></span></div><div class="separator" style="clear: both; text-align: center;"><span style="font-family: Verdana, sans-serif;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4IpLLU3Izlkh19MMgApra4jyVhVW8TvYin-NrLgTNroCIvwatyyDR3ZSCFhR0yA0rP3D3VmRcB_fPndG8ldaQc9YWdPordqmVQo-i5ymlgaRVzQHluN6wQmE2TRmNI_R-qlaSg9Y30IM/s1600/LeanArchitecture.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4IpLLU3Izlkh19MMgApra4jyVhVW8TvYin-NrLgTNroCIvwatyyDR3ZSCFhR0yA0rP3D3VmRcB_fPndG8ldaQc9YWdPordqmVQo-i5ymlgaRVzQHluN6wQmE2TRmNI_R-qlaSg9Y30IM/s320/LeanArchitecture.jpg" width="256" /></a></span></div><span style="font-family: Verdana, sans-serif;"><o:p></o:p></span><br /><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">The first key idea of “Lean Architecture” is the reconciliation of agile and architecture, because of the increase of scale for projects that agile methods are addressing today. “<i>Extreme Programming (XP) started out in part by consciously trying to do exactly the opposite of what conventional wisdom recommended, and in part by limiting itself to small-scale software development. Over time, we have come in full circle, and many of the old practices are being restored, even in the halls and canon of Agiledom</i>”.  On page 161, one finds a nice graph from Boehm and Turner that shows the effect of size on the need for architecture.  As usual with Boehm, this is a data-derived graph that captures one’s intuition:  once a software project becomes large, anticipation and forethinking is required.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">There is no surprise here. On page 15 we read “Ignoring architecture in the long term increases long term costs”, which any one with gray hairs knows firsthand. The insight from this book is that the lean contribution to agile is to bring back the long term and systemic focus into agile, hence the need for architecture.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">Coplien and Bj</span><span lang="EN-US">ø</span><span lang="EN-US">rnvig bring a fresh and simple vision of lean software development, which is summarized in the book by two formulation of the same principle:<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Lean = All hands on deck + long term planning</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Lean = Everybody, All together, Early on</span></li></ul><br /><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">I would not say that “The Toyota Way” could be fit into such an equation, but it captures an important part of what lean management is about, and it’s relevant to emphasize the long term vision which is indeed a key aspect from lean. It also helps the authors to introduce a (creative) tension between lean and agile:</span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Agile is oriented towards change and organic complexity. For instance it is good to defer decision (this is not new or “agile per se” since Knuth is quoted to have said “premature optimization is the root of all evil”).</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Lean is focused towards large-scale complexity (including “complication”)  and long –term resilience. Hence the focus on standardization and bringing decisions forward.</span></li></ul><br /><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">This distinction is presented in a full table page 12, and, frankly, I disagree with most of its rows. For instance, team versus individual is not a good characterization of lean versus agile. Nor is the complicated versus complex debate … and the “high throughput (lean)” versus “low latency (agile)” debate is too restrictive.  My views are expressed <a href="http://informationsystemsbiology.blogspot.fr/2012/12/lean-scrum-agile-and-extreme-programming.html">in a previous post</a> and I think that there is much more in common between lean thinking and agile thinking. Still, I would definitely agree that the management of time, the systemic and holistic view, and the overall long-term perspective is different between lean and agile. Clearly, the goal of lean software development is to combine both, and this is where the extra emphasis on architecture comes from.  It also explains the emphasis on maintainability, as expressed by this quote from Jeffrey Liker about  lean:  </span><span lang="EN-US">“<i>a culture of stopping or slowing down to get quality right the first time to enhance productivity in the long run</i>”. </span><span lang="EN-US">The tension between lean and agile is helpful to explain why one needs both the systemic practice of “<a href="http://en.wikipedia.org/wiki/5_Whys">Five Whys</a>” in the search of root cause while problem solving, together with the frequent feedback and adaptive cycles. Indeed the complexity and the rate of change of our current environment require both: complexity demands the 5 whys, but the rate of change means that this is not enough, quick feedback loops are required.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">An interesting development is this book deals with <a href="http://en.wikipedia.org/wiki/Conway's_law">Conway’s law</a>. This principle states that there is a strong dependency between architecture and organization. That is, the system architecture (module organization) will be strongly influenced by the management organization (teams & departments) that produces the piece of software. Conversely, things work best (from a management perspective) when people organization follows the system architecture. Coplien derives two consequences from this law, which strike me as truly relevant:<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“Manager are über-architects, a responsibility not to be taken lightly” … here goes the myth that one may do a good job at managing software developers without a keen insight about software architecture :)</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span style="font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“There is a need for modularity in large-scale system” – this is obviously true … and indeed a consequence of Conway’s law. The goal of <a href="http://informationsystemsbiology.blogspot.fr/2012/07/lean-software-factory.html">modularity</a>is to keep changes local, and since it is unfortunately mandatory on the people side <a href="http://organisationarchitecture.blogspot.fr/2012/05/larchitecture-organisationnelle-est.html">as soon as the scale of the projects tips on the “large side”</a>, modularity in the organization must translate into modularity in software, hence the need for architecture :)</span></li></ul><br /><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">This post will not do justice to this excellent book, which is full of wisdom. I do not have the time to collect all pearls, such as “<i>Remember that architecture is mainly about people</i>” or “</span><i><span lang="EN-US">Software development is rarely a matter of having enough muscle to get the job done, but rather of having the right skill sets present</span></i><span lang="EN-US">”.  I refer you to my <a href="http://www.amazon.com/Information-Technology-Chief-Executive-Organization/dp/1438911114/ref=sr_1_1?ie=UTF8&qid=1373781086&sr=8-1&keywords=Yves+CASEAU">book</a>for my own two cents of wisdom about information system architecture and management. Still, it worth repeating that architecture is foremost a communication tool to manage change, and not a blueprint for a better word.  One of my favorite quote from “Lean Architecture” is the following: “<i>The customer has a larger stake in the development process than in the service that your software provides</i>”.  This is the crucial idea that is at the heart of the “<a href="http://informationsystemsbiology.blogspot.fr/2012/07/lean-software-factory.html">Lean Software Factory</a>”. In a dynamic and changing world, software is not an object, it is a process. Qualities such as evolvability, modularity and openness come from the people and the development process, much more than they apply to a finished product.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This brings me to my second favorite quote, page 131: “<i>The essence of “Lean” in Lean architecture is to take careful, well-considered analysis and distill it into APIs written in everyday programming language</i>”. Here we see the long-term/resilience thread of lean thinking brought into software development in a very practical manner. This quote expresses two key ideas into one sentence:<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Being “future-proof” is having the right <a href="http://en.wikipedia.org/wiki/Application_programming_interface">APIs</a>so that the code may be both extended easily as well as reused in different manners from which it was intended in the first place.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Defining, I would rather say “growing”, the right set of APIs is an art; it requires practice, wisdom and aesthetical judgment. Still it required forethinking and analysis, which is architectural thinking.</span></li></ul><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This should be enough to reconcile agile programming with architecture, if there ever was a need. Coplien and Bjornvig spend a few delightful pages debunking “Agile Myths” such as:<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ol><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span style="font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“<i>You can always refactor your way to a better architecture</i>”. </span><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">Refactoring is crucial because software is a live object that evolves constantly as its environment changes. Agile methods, like any iterative development processes, are bound to produce “accumulations” that need to be cleaned up.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“<i>We should do things at the last moment</i>”. <br />Here we find the “anticipation versus as-late-as-possible” debate that was mentioned earlier.  Many authors, including <a href="http://www.amazon.com/Lean-Software-Development-Agile-Toolkit/dp/0321150783/ref=sr_1_1?ie=UTF8&qid=1373782899&sr=8-1&keywords=poppendieck">Mary Poppendieck</a>, consider that lean thinking translates into taking design decision as late as possible. This book does a good job at balancing the arguments. There is no single answer, but there is a strong plea for “thinking ahead” and “preparing oneself” through architectural design.</span></li><li><i style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Agile = “don’t do documentation”</span></i><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">. <br />The agile tradition of no comprehensive documentation that gets obsolete before it is used still stands. However, as soon as scale grows, and as soon as the life expectancy of the software piece grows, there is a need for serious documentation. There are many tools to automate part of the documentation task, especially that which is closely linked to the code. A large piece of software requires storytelling, and this has nothing to do with software, it’s a consequence of human nature and what is needed to motivate a large group of people (see <a href="http://www.amazon.com/Whole-New-Mind-Right-Brainers-Future/dp/1594481717/ref=sr_1_1?ie=UTF8&qid=1373783355&sr=8-1&keywords=a+whole+new+mind">Daniel Pink</a>).</span></li></ol><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1H7bOxV4AB4JeoUCYwfeo9Fn4xPXseHhWMlvY3UzrGZ2KDFsQTu9NT_XyuJzLCwjRSzZhlN24n_UB78f-uXq6v_p7WUUxOr8TPKFGN9BwxIGP7bFz6tDgniTyJs5LQd8Ld_Wg3WyuYaA/s1600/UrbanisationSoaBpm4.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1H7bOxV4AB4JeoUCYwfeo9Fn4xPXseHhWMlvY3UzrGZ2KDFsQTu9NT_XyuJzLCwjRSzZhlN24n_UB78f-uXq6v_p7WUUxOr8TPKFGN9BwxIGP7bFz6tDgniTyJs5LQd8Ld_Wg3WyuYaA/s1600/UrbanisationSoaBpm4.jpg" style="cursor: move;" /></a><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">User stories are an integral part of Agile/SCRUM development methods. There are a few interesting pages in this book that show the link between user stories and business processes, which are commonly associated to large-scale waterfall development process.  The last part of the book deals with <a href="http://en.wikipedia.org/wiki/Data,_context_and_interaction">DCI</a>(Data, Context and Interaction), a framework that extends MVC and which is proposed as the proper foundation for designing modern distributed object-oriented systems. This topic is out of the scope of this post, although I find many similarities with the design philosophy of the <a href="http://en.wikipedia.org/wiki/Claire_(programming_language)">CLAIRE programming language</a>. Some of the key insights may be summarized as the need to recognize and separate the need for complex algorithm from object classes (“<i>Does this mean that procedural design is completely dead ? Of course not. There are algorithms that are just algorithms</i>”), the reification of roles and business processes and the use of context objects to develop functional polymorphism (hence to share and reuse more code), a practice that reminds me of my early day at Bouygues Telecom when we created the PTEs (<i>Processus Techniques Elementaires</i>) – cf. my <a href="http://www.amazon.fr/Urbanisation-SOA-BPM-%C3%A9dition-point/dp/2100566369/ref=sr_1_1?ie=UTF8&qid=1373784963&sr=8-1&keywords=Urbanisation+Caseau">book on Urbanization</a>.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">I will conclude with two quotes from page 92:<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“Let the human considerations drive the partitioning [architecture], with software engineering concerns secondary”.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span style="font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“A complex system might use several paradigms”.</span></li></ul><br /> <br /><div class="MsoNormal" style="margin-bottom: 0.0001pt;"></div><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><br /></span></div><span lang="EN-US"><div style="text-align: justify;"><br /></div><!--[if !supportLineBreakNewLine]--><!--[endif]--><o:p></o:p></span></div>

Follow-up on Lean Architecture

This post is a sequel to the previous post regarding the lean architect . It’s main topic is the book review of “ Lean Architecture – for A...

Read more »