Follow-up on Lean Architecture

<div dir="ltr" style="text-align: left;" trbidi="on"><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This post is a sequel to the previous post <a href="http://informationsystemsbiology.blogspot.fr/2013/01/lean-architect-do-we-need-abstraction.html">regarding the lean architect</a>. It’s main topic is the book review of “<b>Lean Architecture</b> – <i>for Agile Software Development</i>”  from James O. Coplien & Gertrud </span><span lang="EN-US" style="font-family: Verdana, sans-serif;">Bj</span><span lang="EN-US" style="font-family: Verdana, sans-serif;">ø</span><span lang="EN-US" style="font-family: Verdana, sans-serif;">rnvig</span><span style="font-family: Verdana, sans-serif;">  This is really a follow-up in the sense that I have found most of the ideas from my previous post expressed in this book, but they are more thoughtfully presented :).  It also goes further than my previous analysis, which is why I am writing this quick – and incomplete – book review. James Coplien  is both a prolific author and a serious expert on software architecture, with an itinerary that is not so different from mine, especially with respect to object-oriented programming, which use to be my own domain of expertise twenty years ago. Similarly <a href="http://informationsystemsbiology.blogspot.fr/2009/04/selected-bibliography.html">the 10 years which I have spent working on information systems architecture</a>from 1997 to 2007, are well mirrored by Coplien track record in system’s architecture in the 90s and 00s.</span></span></div><div class="separator" style="clear: both; text-align: center;"><span style="font-family: Verdana, sans-serif;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4IpLLU3Izlkh19MMgApra4jyVhVW8TvYin-NrLgTNroCIvwatyyDR3ZSCFhR0yA0rP3D3VmRcB_fPndG8ldaQc9YWdPordqmVQo-i5ymlgaRVzQHluN6wQmE2TRmNI_R-qlaSg9Y30IM/s1600/LeanArchitecture.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4IpLLU3Izlkh19MMgApra4jyVhVW8TvYin-NrLgTNroCIvwatyyDR3ZSCFhR0yA0rP3D3VmRcB_fPndG8ldaQc9YWdPordqmVQo-i5ymlgaRVzQHluN6wQmE2TRmNI_R-qlaSg9Y30IM/s320/LeanArchitecture.jpg" width="256" /></a></span></div><span style="font-family: Verdana, sans-serif;"><o:p></o:p></span><br /><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">The first key idea of “Lean Architecture” is the reconciliation of agile and architecture, because of the increase of scale for projects that agile methods are addressing today. “<i>Extreme Programming (XP) started out in part by consciously trying to do exactly the opposite of what conventional wisdom recommended, and in part by limiting itself to small-scale software development. Over time, we have come in full circle, and many of the old practices are being restored, even in the halls and canon of Agiledom</i>”.  On page 161, one finds a nice graph from Boehm and Turner that shows the effect of size on the need for architecture.  As usual with Boehm, this is a data-derived graph that captures one’s intuition:  once a software project becomes large, anticipation and forethinking is required.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">There is no surprise here. On page 15 we read “Ignoring architecture in the long term increases long term costs”, which any one with gray hairs knows firsthand. The insight from this book is that the lean contribution to agile is to bring back the long term and systemic focus into agile, hence the need for architecture.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">Coplien and Bj</span><span lang="EN-US">ø</span><span lang="EN-US">rnvig bring a fresh and simple vision of lean software development, which is summarized in the book by two formulation of the same principle:<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Lean = All hands on deck + long term planning</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Lean = Everybody, All together, Early on</span></li></ul><br /><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">I would not say that “The Toyota Way” could be fit into such an equation, but it captures an important part of what lean management is about, and it’s relevant to emphasize the long term vision which is indeed a key aspect from lean. It also helps the authors to introduce a (creative) tension between lean and agile:</span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Agile is oriented towards change and organic complexity. For instance it is good to defer decision (this is not new or “agile per se” since Knuth is quoted to have said “premature optimization is the root of all evil”).</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Lean is focused towards large-scale complexity (including “complication”)  and long –term resilience. Hence the focus on standardization and bringing decisions forward.</span></li></ul><br /><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">This distinction is presented in a full table page 12, and, frankly, I disagree with most of its rows. For instance, team versus individual is not a good characterization of lean versus agile. Nor is the complicated versus complex debate … and the “high throughput (lean)” versus “low latency (agile)” debate is too restrictive.  My views are expressed <a href="http://informationsystemsbiology.blogspot.fr/2012/12/lean-scrum-agile-and-extreme-programming.html">in a previous post</a> and I think that there is much more in common between lean thinking and agile thinking. Still, I would definitely agree that the management of time, the systemic and holistic view, and the overall long-term perspective is different between lean and agile. Clearly, the goal of lean software development is to combine both, and this is where the extra emphasis on architecture comes from.  It also explains the emphasis on maintainability, as expressed by this quote from Jeffrey Liker about  lean:  </span><span lang="EN-US">“<i>a culture of stopping or slowing down to get quality right the first time to enhance productivity in the long run</i>”. </span><span lang="EN-US">The tension between lean and agile is helpful to explain why one needs both the systemic practice of “<a href="http://en.wikipedia.org/wiki/5_Whys">Five Whys</a>” in the search of root cause while problem solving, together with the frequent feedback and adaptive cycles. Indeed the complexity and the rate of change of our current environment require both: complexity demands the 5 whys, but the rate of change means that this is not enough, quick feedback loops are required.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">An interesting development is this book deals with <a href="http://en.wikipedia.org/wiki/Conway's_law">Conway’s law</a>. This principle states that there is a strong dependency between architecture and organization. That is, the system architecture (module organization) will be strongly influenced by the management organization (teams & departments) that produces the piece of software. Conversely, things work best (from a management perspective) when people organization follows the system architecture. Coplien derives two consequences from this law, which strike me as truly relevant:<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“Manager are über-architects, a responsibility not to be taken lightly” … here goes the myth that one may do a good job at managing software developers without a keen insight about software architecture :)</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span style="font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“There is a need for modularity in large-scale system” – this is obviously true … and indeed a consequence of Conway’s law. The goal of <a href="http://informationsystemsbiology.blogspot.fr/2012/07/lean-software-factory.html">modularity</a>is to keep changes local, and since it is unfortunately mandatory on the people side <a href="http://organisationarchitecture.blogspot.fr/2012/05/larchitecture-organisationnelle-est.html">as soon as the scale of the projects tips on the “large side”</a>, modularity in the organization must translate into modularity in software, hence the need for architecture :)</span></li></ul><br /><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">This post will not do justice to this excellent book, which is full of wisdom. I do not have the time to collect all pearls, such as “<i>Remember that architecture is mainly about people</i>” or “</span><i><span lang="EN-US">Software development is rarely a matter of having enough muscle to get the job done, but rather of having the right skill sets present</span></i><span lang="EN-US">”.  I refer you to my <a href="http://www.amazon.com/Information-Technology-Chief-Executive-Organization/dp/1438911114/ref=sr_1_1?ie=UTF8&qid=1373781086&sr=8-1&keywords=Yves+CASEAU">book</a>for my own two cents of wisdom about information system architecture and management. Still, it worth repeating that architecture is foremost a communication tool to manage change, and not a blueprint for a better word.  One of my favorite quote from “Lean Architecture” is the following: “<i>The customer has a larger stake in the development process than in the service that your software provides</i>”.  This is the crucial idea that is at the heart of the “<a href="http://informationsystemsbiology.blogspot.fr/2012/07/lean-software-factory.html">Lean Software Factory</a>”. In a dynamic and changing world, software is not an object, it is a process. Qualities such as evolvability, modularity and openness come from the people and the development process, much more than they apply to a finished product.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This brings me to my second favorite quote, page 131: “<i>The essence of “Lean” in Lean architecture is to take careful, well-considered analysis and distill it into APIs written in everyday programming language</i>”. Here we see the long-term/resilience thread of lean thinking brought into software development in a very practical manner. This quote expresses two key ideas into one sentence:<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Being “future-proof” is having the right <a href="http://en.wikipedia.org/wiki/Application_programming_interface">APIs</a>so that the code may be both extended easily as well as reused in different manners from which it was intended in the first place.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Defining, I would rather say “growing”, the right set of APIs is an art; it requires practice, wisdom and aesthetical judgment. Still it required forethinking and analysis, which is architectural thinking.</span></li></ul><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">This should be enough to reconcile agile programming with architecture, if there ever was a need. Coplien and Bjornvig spend a few delightful pages debunking “Agile Myths” such as:<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ol><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span style="font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“<i>You can always refactor your way to a better architecture</i>”. </span><span style="font-family: Verdana, sans-serif; text-indent: -18pt;">Refactoring is crucial because software is a live object that evolves constantly as its environment changes. Agile methods, like any iterative development processes, are bound to produce “accumulations” that need to be cleaned up.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“<i>We should do things at the last moment</i>”. <br />Here we find the “anticipation versus as-late-as-possible” debate that was mentioned earlier.  Many authors, including <a href="http://www.amazon.com/Lean-Software-Development-Agile-Toolkit/dp/0321150783/ref=sr_1_1?ie=UTF8&qid=1373782899&sr=8-1&keywords=poppendieck">Mary Poppendieck</a>, consider that lean thinking translates into taking design decision as late as possible. This book does a good job at balancing the arguments. There is no single answer, but there is a strong plea for “thinking ahead” and “preparing oneself” through architectural design.</span></li><li><i style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Agile = “don’t do documentation”</span></i><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">. <br />The agile tradition of no comprehensive documentation that gets obsolete before it is used still stands. However, as soon as scale grows, and as soon as the life expectancy of the software piece grows, there is a need for serious documentation. There are many tools to automate part of the documentation task, especially that which is closely linked to the code. A large piece of software requires storytelling, and this has nothing to do with software, it’s a consequence of human nature and what is needed to motivate a large group of people (see <a href="http://www.amazon.com/Whole-New-Mind-Right-Brainers-Future/dp/1594481717/ref=sr_1_1?ie=UTF8&qid=1373783355&sr=8-1&keywords=a+whole+new+mind">Daniel Pink</a>).</span></li></ol><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1H7bOxV4AB4JeoUCYwfeo9Fn4xPXseHhWMlvY3UzrGZ2KDFsQTu9NT_XyuJzLCwjRSzZhlN24n_UB78f-uXq6v_p7WUUxOr8TPKFGN9BwxIGP7bFz6tDgniTyJs5LQd8Ld_Wg3WyuYaA/s1600/UrbanisationSoaBpm4.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1H7bOxV4AB4JeoUCYwfeo9Fn4xPXseHhWMlvY3UzrGZ2KDFsQTu9NT_XyuJzLCwjRSzZhlN24n_UB78f-uXq6v_p7WUUxOr8TPKFGN9BwxIGP7bFz6tDgniTyJs5LQd8Ld_Wg3WyuYaA/s1600/UrbanisationSoaBpm4.jpg" style="cursor: move;" /></a><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">User stories are an integral part of Agile/SCRUM development methods. There are a few interesting pages in this book that show the link between user stories and business processes, which are commonly associated to large-scale waterfall development process.  The last part of the book deals with <a href="http://en.wikipedia.org/wiki/Data,_context_and_interaction">DCI</a>(Data, Context and Interaction), a framework that extends MVC and which is proposed as the proper foundation for designing modern distributed object-oriented systems. This topic is out of the scope of this post, although I find many similarities with the design philosophy of the <a href="http://en.wikipedia.org/wiki/Claire_(programming_language)">CLAIRE programming language</a>. Some of the key insights may be summarized as the need to recognize and separate the need for complex algorithm from object classes (“<i>Does this mean that procedural design is completely dead ? Of course not. There are algorithms that are just algorithms</i>”), the reification of roles and business processes and the use of context objects to develop functional polymorphism (hence to share and reuse more code), a practice that reminds me of my early day at Bouygues Telecom when we created the PTEs (<i>Processus Techniques Elementaires</i>) – cf. my <a href="http://www.amazon.fr/Urbanisation-SOA-BPM-%C3%A9dition-point/dp/2100566369/ref=sr_1_1?ie=UTF8&qid=1373784963&sr=8-1&keywords=Urbanisation+Caseau">book on Urbanization</a>.<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">I will conclude with two quotes from page 92:<o:p></o:p></span></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“Let the human considerations drive the partitioning [architecture], with software engineering concerns secondary”.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span style="font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">“A complex system might use several paradigms”.</span></li></ul><br /> <br /><div class="MsoNormal" style="margin-bottom: 0.0001pt;"></div><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><br /></span></div><span lang="EN-US"><div style="text-align: justify;"><br /></div><!--[if !supportLineBreakNewLine]--><!--[endif]--><o:p></o:p></span></div>

Follow-up on Lean Architecture

This post is a sequel to the previous post regarding the lean architect . It’s main topic is the book review of “ Lean Architecture – for A...

Read more »

Systemic Simulation of Smart Grids (S3G) - Part III

<div dir="ltr" style="text-align: left;" trbidi="on"><div class="RoadefTitre" style="margin: 24pt 0cm 18pt; text-align: justify;"><span style="font-family: Verdana, sans-serif; font-size: 10pt;">This post concludes the first phase of my computational experiments with S3G (Systemic Simulation of </span><a href="http://www.smartgrid.gov/the_smart_grid" style="font-family: Verdana, sans-serif; font-size: 10pt;">Smart Grids</a><span style="font-family: Verdana, sans-serif; font-size: 10pt;">) which I run during 2011 & 2012 summers. I presented the results at the </span><a href="http://roadef2013.utt.fr/index.htm" style="font-family: Verdana, sans-serif; font-size: 10pt;">2013 ROADEF conference</a><span style="font-family: Verdana, sans-serif; font-size: 10pt;"> a few months ago and I have made the extended set of slides available </span><a href="https://www.box.com/s/e3vm657jfu6kzc8m399p" style="font-family: Verdana, sans-serif; font-size: 10pt;">on my box account</a><span style="font-family: Verdana, sans-serif; font-size: 10pt;"> (left of the blog page – My Shared Documents).</span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><span lang="EN-US" style="font-family: Verdana, sans-serif;"><b><span style="font-size: large;"> 1. S3G Experiments</span></b></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><br /></div><div class="RoadefTexte" style="text-align: justify; text-indent: 0cm;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">A simple description of S3G is available in <a href="http://informationsystemsbiology.blogspot.fr/2012/09/systemic-simulation-of-smart-grids.html">a previous post</a>. It should help to understand what is presented in the slides, since many of those slides were included in that post. </span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">The objective of S3G is to simulate the production and consumption of electricity throughout a long period of time (15 years) with a global “ecosystem” perspective.</span><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-ansi-language: EN-US;"><o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">I will first add some explanations to three topics: the set of models, the satisfaction model and the GTES search for equilibriums. It is important to understand the limits of the current experiment before giving out the preliminary findings, since they need to be taken “with a grain of salt”.<o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><div style="text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">As mentioned earlier, S3G uses very simple models (i.e. simple equations and few parameters) for the component of the “energy production & consumption” complex system. This is a deliberate choice, because I lack the expertise to produce more complex sub-models, and, mostly, because I want to focus on the overall system complexity (that is, what happens when all this simple subsystems are put together). This is clearly explained in my <a href="http://www.dailymotion.com/video/xxvknb_intervention-yves-caseau-inauguration-irt-systemx_tech#.UWo8dqe-18E">SystemX IRT introduction keynote</a>. Still, it’s worth taking a look at each of these sub-models:<o:p></o:p></span></div></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"></div><ul style="text-align: left;"><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">The energy demand generation is quite simple. I start with daily and yearly patterns, obtained by cut & pasting historic curves found on the web and I had random noise, which I can control (time or geography-dependent). I don’t think that this is a limitation for this experience.</span></li><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">I have a crude vision of “NegaWatts” that represent energy consumption that may be saved through energy saving investment. NegaWatts are virtuous: there is no reduction of economic output, but they require money. Here my model is really too simple, but somehow it falls outside the scope of what I was trying to accomplish. I use a simple hyperbolic function to represent the fact that, as electricity prices grow, people are likely to invest to try to reduce their consumption. Since it is very difficult to foresee the negawatt development in the next 20 years, it is better to use a single parameter (slope of the hyperbole) and make it vary to cover all kinds of scenarios.</span></li><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">I have an equally crude model for demand/response, which, contrary to Negawatts, is instantaneous but affect economic output. In my model (a simple S-curve), demand is reduced when peak price becomes too high. We’ll see later on that this is indeed too simple and that it should be further developed in a future next step.</span></li><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">The market share model – to determine the market share of the grid operator against the incumbent – is a simple/classic S-curve. My <a href="http://organisationarchitecture.blogspot.fr/2008/01/lapproche-gtes-et-les-quilibres-entre.html">previous experience with similar economic simulation</a> tells me that it is enough to produce a realistic experience (this is not the the systemic complexity lies).</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">On the other hand, the dynamic pricing model - how does the incumbent modulate his wholesale/retail price - is the heart of the relationship between the local and the national operators, and my current version is too simple. I assume that the price is a function of the output (demand), so that peak consumption yields a higher price. I have chosen a very simple function for my dynamic price equation: a piece-wise linear function, with a constant price up to a fixed (constant) production, and then a linear surcharge when the production is higher than this constant. Obviously, one would like to test and analyze more complex dynamic pricing schemes, since dynamic price and demand/response behaviors <a href="http://informationsystemsbiology.blogspot.fr/2011/09/systemic-simulation-of-smart-grids.html">are a key engine for smart grids</a>. The reason why I used a simple model is that this is precisely the complexity-generator for the model, and using a randomly-complex model makes it very difficult to analyze later. This pricing structure is under the control of regulation, and I am waiting to better understand what our political instances have in mind to encode a richer model (see later).</span></li><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">Last, the smart grid electricity production model is reasonably detailed for such an experiment. The decision about which source of electricity to use is actually straightforward due to the mix of production constraint (one must use the electricity that is produced) and economic goals (when sourcing, get the cheaper source). The only tricky part is the management of storage. I use simple rules, with a number of parameters that are tuned within the machine learning loop of the GTES simulation. Hence I let the simulation engine discover how to best use local storage. For instance, the local operator can both use storage as a “buffer” for its own production or as “reserve” to play the market (buy when cheap and sell when expensive). When I consider the small amount of storage that is actually used (because of storage price), this part of the model is quite satisfactory.</span></li></ul><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">A GTES run is the simulation of an optimization loop that tries both to maximize each player’s satisfaction and to find a Nash equilibrium. Hence, defining each player’s satisfaction is a critical part of this S3G. Let us first recall that there are four players (actually, set of players) in this “game”. Each player has three goals, with a target value that is associated to each goal. We define a “strategy” as a triplet of pairs (goal, target). The satisfaction is then expressed with respect to each goal with a pseudo-linear function: 100% if the target value is reached and a linear fraction otherwise. The overall satisfaction for a strategy is the average satisfaction with respect to the three goals of the strategy. In the GTES method, we separate the parameters that represent these goals (grouped as a strategy) from the other parameters that the player may change to adjust its results, which we call the “tactical play”. The principle of the GTES game is that each player tries to adjust its “tactical parameters” to maximize its “satisfaction” (w.r.t its strategy). Here is a short description of the four players in the S3G game:<o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -18pt;">The “<b>regulator</b>” (political power) whose goal is to reduce CO2 emissions while preserving economic output and keeping a balanced budget (between taxes and incentives). Its three “goals” are, therefore, the total output (consumed electricity + negaWatts), the amount of CO2 and the state budget (taxes - subsidies). Its tactical play includes setting up a CO2 tax, regulating the wholesale price for the suppliers and creating a discount incentive for renewable energies.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -18pt;">The existing energy companies, here called “<b>suppliers</b>”, whose goal is to maintain their market-share against newcomers, maintain revenue and reduce exposure to consumption peaks. Their tactical play is mostly through pricing (dynamic), but they also control investment into new production facilities on a yearly basis.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -18pt;">The new local energy <b>operators</b>, who see “smart grids” as a differentiating technology to compete against incumbents. Their goal is to grow turnover, EBITDA and market-share. Their real-time tactical play is dynamic pricing, and they may invest into renewable and fossil energy production units, as well as storage units.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -18pt;">The consumers are grouped into <b>cities</b>, whose goal is to procure electricity at the lowest average price, while avoiding peak prices and preserving their comfort. The cities’ tactical play is mostly to switch its energy supplier (on a yearly basis) and to invest into “negaWatts”, which are energy-saving-investments (more energy-efficient homes, etc.).</span></li></ul><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><b><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">GTES</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;"> stands for Game-Theoretical Evolutionary Simulation. I have talked about it in <a href="http://organisationarchitecture.blogspot.fr/search?q=GTES">various posts</a>, and a summary may be found <a href="http://claire3.free.fr/yves_research.htm">here</a>. I gave a keynote talk about GTES at <a href="http://www.amazon.com/Complex-Systems-Design-Management-International/dp/3642344038/ref=sr_1_1?ie=UTF8&qid=1367745267&sr=8-1&keywords=CSDM+2012">CSDM 2012</a>, the slides are available <a href="http://www.csdm2012.csdm.fr/IMG/pdf/Y_CASEAU-_Game_Theoretical_and_Evolutionary_Simulation_a_toolbox_for_complex_enterprise_problems_ok-2.pdf">here</a>. </span><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-ansi-language: EN-US;">GTES is a framework designed to study a model through simulation, in order to extract a few properties from this model (learning through examples), either explicitly or implicitly.  GTES is based upon the combination of three techniques:<o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ol><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -17.85pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt; line-height: normal;"> </span></span><u style="line-height: 12pt; text-align: left; text-indent: -17.85pt;"><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-fareast-language: EN-US;">Sampling</span></u><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -17.85pt;">:  since some parameters that occur in the economic equations are unknown, we draw them randomly from a confidence interval, using a <a href="http://en.wikipedia.org/wiki/Monte_Carlo_method">Monte-Carlo approach</a>. Monte-Carlo simulation has become quite popular (especially in the finance world) over the last decades (while computers became more powerful, obviously). <b>The need for Monte-Carlo is a signature of complexity and non-linearity</b>: simulation becomes necessary when one cannot reason with averages. The beauty of linear equations is precisely that one may work the average value. In a complex non-linear system, deviations are amplified and there is no other way to predict their effect than to look at it, case by case (hence the <i>sampling approach</i>).</span></li><li><u style="line-height: 12pt; text-align: left; text-indent: -17.85pt;"><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-fareast-language: EN-US;">Search for Nash Equilibrium in a repeated game</span></u><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -17.85pt;">: We set the parameters that define the player’s objective functions and <a href="http://arnetminer.org/publication/gtes-une-m-thode-de-simulation-par-jeux-et-apprentissage-pour-l-analyse-des-syst-mes-d-acteurs-3139251.html">look for an equilibrium using an iterative fixed-point approach</a> (in the tradition of the <a href="http://en.wikipedia.org/wiki/Cournot_competition">Cournot Adjustment</a>). The good news with S3G is that it is a “simple” complex system, hence finding a Nash equilibrium is easy. However, it is precisely easy because of the simple pricing model (cf. previous discussion).</span></li><li><u style="line-height: 12pt; text-align: left; text-indent: -17.85pt;"><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-fareast-language: EN-US;">Local Search as a machine learning technique</span></u><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -17.85pt;">: once the parameters that define the objective function are set, the other parameters that define the behavior of each player may be computed to find each player’s “best response” to the current situation. We use a simple local search (“local moves” = dichotomic search for the best value for each tactical parameter), coupled with “<a href="http://en.wikipedia.org/wiki/2-opt">2-opt</a>” : the random exploration of moving two parameters at the same time, using “hill climbing” as a meta-heuristic. From an OR point of view, this are rudimentary techniques, but they seem to do the job. The complexity of the optimization engine that one must embed into GTES depends on the complexity of the model. If the dynamic pricing model was made more complex, a stronger local search <a href="http://en.wikipedia.org/wiki/Metaheuristic">metaheuristic</a> would be necessary.</span></li></ol><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span style="font-family: Verdana, sans-serif; font-size: 10pt;">Explaining </span><b style="font-family: Verdana, sans-serif; font-size: 10pt;">GTES</b><span style="font-family: Verdana, sans-serif; font-size: 10pt;">will take many years … I was invited at </span><a href="http://www.roadef.org/content/roadef/soireeRO.htm" style="font-family: Verdana, sans-serif; font-size: 10pt;">ROADEF’s yearly event</a><span style="font-family: Verdana, sans-serif; font-size: 10pt;">last month to present some of the successes that I have had with this approach over the past 10 years. I have a book, “</span><b style="font-family: Verdana, sans-serif; font-size: 10pt;">Enterprises and Complexity: Simulation, Games and Learning</b><span style="font-family: Verdana, sans-serif; font-size: 10pt;">” in my “pipe”, but I expect at least five more years of work are needed to get it to a decent state (in terms of ease of understanding).</span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><span lang="EN-US" style="font-family: 'Times New Roman', serif; font-size: 13.5pt;"><br /></span><b><span lang="EN-US"><span style="font-family: Verdana, sans-serif; font-size: large;">2. Most interesting findings with S3G experiments</span><span style="font-family: 'Times New Roman', serif; font-size: 16pt;"><o:p></o:p></span></span></b></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><br /></div><div class="MText" style="line-height: 12.0pt; text-indent: 0cm;"><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-fareast-language: EN-US;">A “S3G session” is made of interactive runs of “experiments”, which are GTES computational executions. More precisely, an experiment is defined through two things:<o:p></o:p></span></div><div class="MText" style="line-height: 12.0pt; margin-left: 46.4pt; mso-list: l4 level1 lfo3; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10.0pt; mso-fareast-language: EN-US;">-<span style="font-size: 7pt; line-height: normal;">          </span></span><!--[endif]--><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-fareast-language: EN-US;">The randomization boundaries, for those parameters that will be sampled.<o:p></o:p></span></div><div class="MText" style="line-height: 12.0pt; margin-left: 46.4pt; mso-list: l4 level1 lfo3; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-size: 10.0pt; mso-fareast-language: EN-US;">-<span style="font-size: 7pt; line-height: normal;">          </span></span><!--[endif]--><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-fareast-language: EN-US;">Some specific values for some parameters, since the goal of a “serious game” is to play “what-if scenarios”, by explicitly changing these parameters. For instance, we may play with the investment cost of storage, to see if storage is or will be critical to smart grids.<o:p></o:p></span></div><div class="MText" style="line-height: 12.0pt; margin-left: 46.4pt; mso-list: l4 level1 lfo3; text-indent: -18.0pt;"><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-fareast-language: EN-US;"><br /></span></div><div class="MText" style="line-height: 12pt; text-align: justify; text-indent: 0cm;"><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-fareast-language: EN-US;">Multiple scenarios have been played to evaluate the sensitivity to “environmental parameters such as the variability of energy consumption (globally or locally), the fossil energy price (gas and coal), the possible reduction of the nuclear assets, the impact of carbon taxes or the impact of wholesale price regulation. Here is a short summary of the main findings that were presented at ROADEF:</span></div><div class="MText" style="line-height: 12pt; text-align: justify; text-indent: 0cm;"></div><ul><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><b style="line-height: 12pt; text-align: left; text-indent: -18pt;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">Smart Grids and variability</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-align: left; text-indent: -18pt;">.</span><span style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">One theoretical advantage of smart grids operator is that they </span><a href="http://large.stanford.edu/courses/2011/ph240/bogdanowicz1/" style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">could react better to variations</a><span style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">. Simulation does show some form of better reaction from the local operator than the national operator to either fluctuation (electricity demand that varies compared to historical forecast) and local variation (for instance, through local changes of climatic conditions). However the difference is very small, and could be disqualified as insignificant from a statistical perspective. This result depends on storage price (see later) and wholesale price structure. With the current values, one of my key arguments in favor of smart grids (systems of distributed systems are expected to be more flexible and reactive) does not seem to hold.</span></li><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-align: left; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><b style="text-align: left; text-indent: -18pt;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">Carbon tax and Nuclear strategy</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">.<br /><span style="font-size: 10pt; text-indent: -18pt;">I played with carbon tax to see </span><a href="http://theenergycollective.com/jeffstjohn/208331/how-carbon-reduction-and-smart-grid-work-together" style="font-size: 10pt; text-indent: -18pt;">if the raise of carbon tax would have an effect</a><span style="font-size: 10pt; text-indent: -18pt;">, and it does, but it is a negative one since it favors nuclear energy and since green energy is still too expensive. On the other hand, the decision to reduce the share of nuclear energy in the national supplier (either for a long-term withdrawal or a long-term cap as announced by the French government) </span><a href="http://www.memoori.com/trouble-with-nuclear-power-reinforces-the-need-for-smart-grid/" style="font-size: 10pt; text-indent: -18pt;">creates favorable conditions for smart grid operators</a><span style="font-size: 10pt; text-indent: -18pt;">, quite logically. However, simulation shows that the results are weak (small advantage) and unstable (they depend heavily on the overall systemic equation of wholesale prices coupled with “environment” variables such as energy prices).</span></span></li><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-align: left; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><b style="text-align: left; text-indent: -18pt;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">Storage and Photo-voltaic costs</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">.</span><span style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">As explained earlier, I used the Web as my main information source, and got unit prices for storage (per </span>MWh<span style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">) and </span>photo-voltaic<span style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;"> that vary considerably according to the sources. I designed a number of scenarios to see what would happen if the prices fall down, as is expected by a number of “green experts”. </span><a href="http://smartgrid.ieee.org/august-2012/645-what-next-for-energy-storage" style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">The availability of cheap storage has an important impact</a><span style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">, but one need to see a price reduction by a ratio of 5 to 10 (depending where you place the start point) to see this impact materialize. The simple rule seems to be that storage </span>TCO<span style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;"> (total cost of ownership) should get as low as 50% of wholesale price to shift the system’s behavior (quite logical if you think about it). A similar remark may be made about Photo-voltaic energy, which price is still far too high to change the smart grid operator economic mode.</span></li><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-align: left; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;">  </span></span><b style="text-align: left; text-indent: -18pt;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">Wholesale & retail price structure</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">.</span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">This is the heart of the smart grid ecosystem: the rules/regulation that governs wholesale pricing – which controls the “<a href="http://www.globalforumblog.com/session-9">coopetitio<span style="color: black;">n</span></a>” between supplier & operator – and the dynamic pricing for retail, which controls the benefits driven both from demand/response and negawatts.</span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;"> In the game theory tradition, we have built a strategy matrix that shows the result of conflicting strategies between the supplier and the operators, ranging from bold (focus on market-share) to aggressive (focus on revenue) through “soft” (more conservative). The sensitivity to the price regulation structure is such that it does not make sense to draw too much out of my simulations, except the fact that this is the critical part.</span></li><li><b style="text-align: left; text-indent: -18pt;"><span lang="EN-US" style="font-family: "Verdana","sans-serif"; font-size: 10.0pt; mso-ansi-language: EN-US;">Sensitivity to oil price</span></b><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">.</span><span style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">I have played with a number of scenario regarding fossil energy price trends in the next 15 years. The sensitivity is much lower than expected, when comparing suppliers against operators. There is a clear impact on consumers, but the benefit in favor of green energy and smart grid operators is offset by the advantage in favor of nuclear energy. One may add that with shale gas, non-conventional oil and coal, this type of scenario is not likely in the next 15 years. </span><a href="http://www.fierceenergy.com/story/us-electric-prices-driving-data-centers/2013-03-21" style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">What we see in the US is precisely the opposite</a><span style="font-family: Verdana, sans-serif; font-size: 10pt; text-align: left; text-indent: -18pt;">.</span></li></ul><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><b><span lang="EN-US"><span style="font-family: Verdana, sans-serif; font-size: large;">3. Limits of current approach and next steps</span></span></b><span lang="EN-US" style="font-family: 'Times New Roman', serif; font-size: 13.5pt;"><br /><!--[if !supportLineBreakNewLine]--><br /><!--[endif]--></span><b><span lang="EN-US" style="font-family: 'Times New Roman', serif; font-size: 12pt;"><o:p></o:p></span></b></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">Let me first summarize three obvious limits to the S3G approach:<o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">As explained, both wholesale and retail dynamic pricing models are too simple. The shape of the curve is simplistic, but also the fact that price only depends on total demand is unrealistic (taking production costs into account is a must).</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">One of the expected benefits of smart grids is improved resilience, both to catastrophic events and to significant internal failures. I did not try to evaluate resilience, because I did not have enough data to generate meaningful scenarios. If you look at what is happening in Japan, <a href="http://theenergycollective.com/katherinetweed/210736/where-s-my-microgrid">local storage is deployed to increase resilience</a> in the advent of a natural catastrophe, with good reasons (together with <a href="http://www.hitachi.com/products/smartcity/solution/hems/index.html">HEMS</a>: home energy management systems).</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">My <a href="http://en.wikipedia.org/wiki/Demand_response">demand/response</a> model is equally too simple, from two separate perspectives. First I only look at “shaving”, that is electricity that is not consumed, because the usage is forsaken for price/availability reasons. Another interesting alternative is to look at demand displacement, where the consumptions is “shifted” instead of “shaved”. Many usages, mostly related to heating, have enough inertia to be shifted by a few minutes. The other simplifying dimension is that I only look at the instantaneous benefit brought by demand/response that is the non-consumption of electricity at a time when prices are high. However, market prices do not raise that much, nor long enough, to make this “shaving” worth a lot of effort. On the other hand, it may help to avoid investing into excessive marginal capacity, which has a higher payoff.</span></li></ul><br /><div class="MsoListParagraphCxSpLast" style="margin-bottom: 0.0001pt; text-indent: -18pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;"><!--[if !supportLineBreakNewLine]--><br /><!--[endif]--><o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">This last argument is pointed out in the « <a href="http://www.actu-environnement.com/ae/news/loi-brottes-marche-capacite-effacement-17576.php4">Loi Brottes </a>». This article explains clearly the difference between the “capacity adjustment” value creation and the “production adjusting”. <o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"></div><ul style="text-align: left;"><li><i><span style="background-color: white;"><span style="font-family: Times, Times New Roman, serif;"> "En l'état actuel du droit, aucun mécanisme n'est prévu pour rémunérer l'effacement au titre de sa valeur en capacité entre fin 2013 et l'hiver 2015-2016, autrement que par le biais du mécanisme d'ajustement, ce qui limite le développement des capacités d'effacement"</span></span></i><span style="background-color: white;"><span style="font-family: Times, Times New Roman, serif;">, indique-t-il dans l'exposé des motifs. Cet amendement a donc pout but d'assurer dès l'entrée en vigueur de la loi<span class="apple-converted-space"><i> </i></span><i>"le développement des effacements par un dispositif d'appel d'offres, dans l'attente de la mise en place du mécanisme de capacité pérenne qui permettra aux acteurs concernés de développer des capacités de production et d'effacement de consommation"</i>.</span></span></li></ul><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt; text-align: justify;"><span lang="EN-US" style="background-color: white; background-position: initial initial; background-repeat: initial initial; font-family: Verdana, sans-serif; font-size: 10pt;">The reason why I focus on « production adjustment » is that it is much easier to simulate. Capacity adjustment is a three-parties value proposition (the user, the demand-response operator, and the producer whose capacity may be reduced). It requires a regulation (hence the Brottes law), to shift the capacity avoidance into operational benefits for the operator who will eventually share it with the user.<o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><br /></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt;">I will leave the S3G code alone for a while. When I resume this work (2014), I plan to take the following next steps:<o:p></o:p></span></div><div class="MsoNormal" style="margin-bottom: 0.0001pt;"></div><ul style="text-align: left;"><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">Improve the satisfaction formula, using a product form instead of a sum. This is a classic technique when defining KPI for performance measurement. A product (i.e., multiplying the various sub-terms of section 1 instead of summing them) yields a more truthful representation of strategic satisfaction (it is much better to reach all three goals at 60% than getting 100% on two and totally missing the third).</span></li><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">Introduce parallelism (with a <a href="http://en.wikipedia.org/wiki/MapReduce">MapReduce</a>architecture) to reach more stable results with more samples. Monte-Carlo simulation is designed for easy parallelization.</span></li><li><span lang="EN-US" style="font-family: Symbol; font-size: 10pt; text-indent: -18pt;"><span style="font-family: 'Times New Roman'; font-size: 7pt;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; text-indent: -18pt;">Enrich the dynamic pricing model (while sticking to piecewise-linear formulas) and re-evaluate the “model constants” (energy production and storage prices, which constanly evolve).</span></li></ul><br /><div class="MsoNormal" style="margin-bottom: 0.0001pt;"><br /></div><div class="RoadefTitre" style="margin-bottom: 18.0pt; margin-left: 0cm; margin-right: 0cm; margin-top: 24.0pt;"><br /></div><br /><div class="RoadefRef"><br /></div></div>

Systemic Simulation of Smart Grids (S3G) - Part III

This post concludes the first phase of my computational experiments with S3G (Systemic Simulation of Smart Grids ) which I run during 2011 ...

Read more »

« The Lean Architect » - Do we need Abstraction and Planning « on the Gemba » ?

<div dir="ltr" style="text-align: left;" trbidi="on"><br /><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="line-height: 115%;">Today’s post is a first attempt at defining the role of a software architect in a “</span><span style="line-height: 115%;"><a href="http://informationsystemsbiology.blogspot.fr/2012/07/lean-software-factory.html"><span lang="EN-US">Lean Software Factory</span></a></span><span lang="EN-US" style="line-height: 115%;">”, that is, an agile development organization built around </span><span style="line-height: 115%;"><a href="http://www.lean-it-summit.com/"><span lang="EN-US">lean software</span></a></span><span lang="EN-US" style="line-height: 115%;"> principles. There is a natural tension between the two words “<b>lean</b>” and “<b>architect</b>”, so one may think that this title is an oxymoron: “lean management”, in general, tries to stay away from abstraction and too much planning, and promotes practical, concrete team-oriented problem solving “<i>on the gemba</i>”, where the action takes place. Agile development says that conception (requirements and design) evolves continuously while a software product is being developed. This is in contradiction with the architect’s role as defined in the <a href="http://en.wikipedia.org/wiki/V-Model_(software_development)">V software development model</a>, which is the product of Taylorism applied to development, where the Architect (capital A) thinks a grand design that is built by others. The team-oriented approach of lean software development relies on democracy and consensus, where each voice is heard, and where no “design architecture” can be accepted as a “non-negotiable” requirement. Similarly, a key practice of lean/agile software development is synchronicity (everyone working on the same “takt time”), which is implemented through stand-up meetings, <a href="http://www.shmula.com/obeya-communication-breakdown/385/">common project rooms</a>and intense real-time discussion with the product owner, which means that the very concept of “planning” and “design” seems in jeopardy. <o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="line-height: 115%;"><br /></span></span></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US" style="line-height: 115%;"><span style="font-family: Verdana, sans-serif;">The first question I’d like to ask is, therefore, “<i>do we need an architectural process when software development is led in a lean/agile mode ?</i>”. My answer is a resounding “YES”, with the following three arguments:<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">Agile or lean software development does not prevent from building software products that are actually complex and require a carefully-crafted form of internal organization. Let us recall that architecture is foremost about communication: its role is to provide organizations and actions with a narrative sense.  In a lean software development process, architecture is a “co-creation”:  architecture and design continuously evolve together with product development; they are not “completed” until the product is.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">In a continuous development and integration process, we need an architectural framework to prevent from diverging (too many additions that move the product in different and opposite directions) and maintain a continuous and efficient <a href="http://en.wikipedia.org/wiki/Code_refactoring">refactoring</a>. Software code is a <i>living </i>object, which evolves together with the rythms of <a href="http://en.wikipedia.org/wiki/Scrum_(development)#Sprint">sprints</a> and update <a href="http://en.wikipedia.org/wiki/Continuous_delivery">deliveries</a>. As a living object, constant “<a href="http://www.craigberntson.com/blog/?p=964">gardening</a>” is required, which is guided by a vision (of what the garden should be), which is precisely what an “architecture” should be. As Jean-Claude Ameisen explains in his wonderful book “<a href="http://lasculptureduvivant.free.fr/">La Sculpture du Vivant</a>”, the birth of a living organism requires as much pruning and removal as creating and addition; this applies to software : there should be as much thoughts and care put into code pruning and removal as there is in lines-of-code addition.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">A key principle from Toyota/Lean Management is the importance of a systemic vision that is shared by all actors (sharing a systemic understanding is a key role of the <a href="http://www.sixthsigma.com/2006/07/toyota-kaizen-obeya-room.htm">Obeya room</a>). Architecture in a lean software development process is not restricted to a few hand-picked specialists; <a href="http://carlokruger.com/?p=10">it is everyone’s responsibility</a>, especially developers’. A major feedback from SCRUM practitioners is the importance of “<a href="http://scribeandscrum.com/tag/design-review-committee/">design reviews</a>”. Organizing successful design reviews is better conducted by “experts”, who are people with experience (architecture is an art, much more than a science – one learns from previous mistakes). These are precisely “architects”, people who are well qualified to lead a successful review and to encourage learning/capitalization. <i>Agile </i>does not mean than everything needs to be rediscovered “on the gemba”; 30 years of software architecture have produced <a href="http://informationsystemsbiology.blogspot.fr/2009/04/selected-bibliography.html">both useful principles and tools</a>.</span></li></ul><br /> <div class="MsoNormal" style="text-align: justify;"><span lang="EN-US" style="line-height: 115%;"><span style="font-family: Verdana, sans-serif;">The second question to ask is then « <i>What role should an architect play in this emergent software development process ?</i> ». Part of the answer is common to the previous question and has been brushed in the previous three bullet points. The architect is required to shepherd the continuous refactoring and to lead the regular <i>design reviews</i>. This is quite different from the classical role as defined by the V-model. Let us emphasize the practical and cultural differences through three critical insights:<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">An architect needs to be a <a href="http://storytellersforgood.com/2010/08/a-new-age-a-whole-new-mind/">story teller</a>. Agile processes suggest working on requirements expressed as “<a href="http://agiletools.wordpress.com/2007/11/24/task-boards-telling-a-compelling-agile-story/">story boards</a>”. Story boards make for “units of requirement” which are gathered into “backlogs”. The “architectural vision” also needs to be expressed as a set of “stories”.  Stories are a great form for expressing a requirement because they both carry meaning and they are robust. The heart of the agile process is that volatility, randomness, complexity of environment will force the product developers to adapt continuously during its completion. Strict and formal requirements tend to become obsolete or incompatible with one another too soon! Stories are robust because they can evolve; the combination of their meaning is context-dependent. I would qualify stories as “</span><span style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;"><a href="http://www.guardian.co.uk/books/2012/dec/16/antifragile-nassim-nicholas-taleb-review"><span lang="EN-US">antifragile</span></a></span><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">” following Nassim Taleb’s insight: the volatility and variations that will necessarily occur will “shake” the story and often produce even more value than the original thought contained. Serendipity is produced by the random shocks between stories from the backlog and continuous feedback from the users.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">An architect works in a “<i>pull mode</i>”, she is pulled by the need for advice, assistance or explanation from developers. This is very different of the “<i>push mode</i>” from the Taylor-style organizations where the architect would hand out a document to the developers: “<i>please implement the following architecture requirement R1 to R112</i>”. This requires some form of <a href="http://www.mindtools.com/pages/article/newPPM_82.htm">change management</a>, but this new way of working is quite fulfilling for an architect. She may feel the pleasure of participating to the actual creation, instead of being frustrated to see her “great principles, well-written in a forward-looking design document” ignored. This is the instantiation of a key lean principle that says that each actor in a process is at the service of the next actor (in the value creation process, from a chronological viewpoint). <a href="http://en.wikipedia.org/wiki/Kanban_(development)">Kanban</a>, both its industrial and <a href="http://en.wikipedia.org/wiki/Kanban_(development)">software version</a>, is the tool that puts this principle into practice.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">Architects are stakeholders in the <a href="http://www.mountaingoatsoftware.com/scrum/product-backlog">SCRUM backlog</a>management: they ensure that performance tuning and bug fixing (especially “non-functional ones”) are maintained on the top of the stack.  Obviously the product owner is the key stakeholder, and these concerns should be hers/his as well. An architect is not a “<a href="http://www.lean.org/shook/displayobject.cfm?o=906">chief engineer</a>” in the Toyota sense (it is unclear to me if there is such a role in a lean software development process, although it is often used in the <a href="http://en.wikipedia.org/wiki/DevOps">Devops</a> context). As with any complex project, there exists a recursive/fractal structure associated with a product (systems and sub-systems). There are a number of “product owners” (for subsystems) and a master “product owner” who plays a role similar to that of a chief engineer. The architect is a member of the engineering team, a voice that reflects the value of long-term thinking, simplicity and “well-oiled” running. Using lean/agile does not remove the age-old conflict between short-term and long-term goals. Products tend to last (at least, customers wish they would), hence long-term characteristics are essential to a well-designed and well-built product.</span></li></ul><br /> <div class="MsoNormal" style="text-align: justify;"><span lang="EN-US" style="line-height: 115%;"><span style="font-family: Verdana, sans-serif;">To conclude this post, I would like to stand back and ask a more general question « <i>What is the role of the architect from a systemic viewpoint ?</i> ».<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ul><li><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">At the side of the product owner, the lean architect carries the critical mission to build the “situation potential” in <a href="http://scienceofstrategy.org/main/content/article-pdf-41-future-potential">Sun Tzu’s</a> sense. The concept of “<a href="http://books.google.fr/books?id=OS_YqZhWuAsC&pg=PA39&lpg=PA39&dq=%22situation+potential%22+Jullien&source=bl&ots=hxZZQxGfaU&sig=hF7CzO0FIExCrhH-lEAPlILwWGo&hl=en&sa=X&ei=X0bpULfFLqWV0QX8mIDAAw&ved=0CDAQ6AEwAA#v=onepage&q=%22situation%20potential%22%20Julli">situation potential</a>” is the cornerstone of François Jullien’s book on efficiency. In a complex world, both <a href="http://organisationarchitecture.blogspot.fr/2009/05/simulation-jeux-et-previsions-face.html">François Jullien and Nassim Taleb</a> tell us that forecasting (as in “strategic planning”) does not work well any longer and one should become “opportunistic”, in the (positive) Chinese sense (i.e., be prepared to jump on opportunities). Being able to leverage the future, whatever it holds, is the heart of this “Art of War” strategy. It relies on agility: to react quickly as soon as the opportunity arises. It also relies on building the right set of capabilities: the situation potential. A “good architecture” is precisely a “situation potential”. This becomes even more obvious as soon as the necessity of a <b>platform strategy</b> occurs – the core message of “<a href="http://sivers.org/book/WhatWouldGoogleDo">What would Google do ?</a>” by Jeff Jarvis. Most successful software products are platforms (cf. the <a href="http://news.ycombinator.com/item?id=3102800">famous post by Steve Yegge</a>). Designing a platform requires every developer to be eager to contribute to an “open API culture”, to be ready  “to eat your own dogfood” (the best way to implement the feedback circle that is necessary to reach quality). However, designing a powerful platform is precisely an architectural task.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">A key tenet of “<a href="http://informationsystemsbiology.blogspot.fr/2011/11/lean-it-devops-and-cloud-programming.html">lean software</a>”, as explained by Mary & Tom <a href="http://www.poppendieck.com/">Poppendieck</a>, is to free up creativity and innovation through cooperation. Architecture may be seen as the “<b>grammar</b>” for this cooperation. This is the core of “service-oriented architecture” (<a href="http://informationsystemsbiology.blogspot.fr/2009/05/soa-tale-of-two-cities.html">SOA</a>). A software architect, irrespectively of the software development method (agile or not), aims to develop <i>sharing </i>and <i>reuse </i>of software components. This brings us back to the critical role of architecture to develop a software platform. This is <a href="http://theleanstartup.com/">lean</a> & <a href="http://www.pretotyping.org/">pretotyping</a> at work: designing a platform is not (only) about thinking right, it is mostly about feedback loops and listening to users/customers. The value of a grammar is judged from the beautiful sentences that other write through it.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">The more evolution we see in our current world, the more we need solid foundations to avoid losing our time building, destroying and re-building over and over. <a href="http://en.wikipedia.org/wiki/Multilayered_architecture">Layered architecture</a> stays a relevant paradigm in the world of “lean software” because layer structure tells us the order in which things should be built and re-built. Experience shows that one cannot rely only of “auto-organization”: too many iterations – that entail massive rework – are necessary until a fixed-point is reached. Also, the practice of 5S (</span><i style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"><span lang="EN-US" style="background-color: white; line-height: 115%;">sorting, set in order, systematic cleaning, standardizing, and sustaining)</span></i><span lang="EN-US" style="background-color: white; font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">, applied to software (as explained by Mary Poppendieck for instance)</span><span lang="EN-US" style="font-family: Verdana, sans-serif; line-height: 115%; text-align: left; text-indent: -18pt;">, is both relevant to code development and software architecture. There are many other interesting architectural contribution to a lean software product, such as the <a href="https://sites.google.com/a/gertrudandcope.com/www/thedciarchitecture">DCI architecture</a>, which may be found in the book “<a href="http://www.leansoftwarearchitecture.com/">Lean Software Architecture</a>”.</span></li></ul><br /> <div class="MsoNormal"><br /></div><div class="MsoNormal" style="text-align: justify;"><span lang="EN-US" style="line-height: 115%;"><span style="font-family: Verdana, sans-serif;">This short post does not do justice to the complex topic of “lean software development” mixed with “software architecture”. The title, “Lean Architect”, applies to a long-term goal, which is to reconcile large-scale-systems-engineering with agile development methods. My own conviction is that “lean” – in the “<a href="http://en.wikipedia.org/wiki/The_Toyota_Way">Toyota Way</a>” sense -  is the appropriate framework for such an integration. Lean adds to agile methods the <a href="http://informationsystemsbiology.blogspot.fr/2012/12/lean-scrum-agile-and-extreme-programming.html">combination of a global/systemic vision</a>, together with a practical focus on concrete issues (“<i>on the <a href="http://en.wikipedia.org/wiki/Gemba">gemba</a></i>”). This is actually the core of Nassim’s Taleb “<a href="http://www.nytimes.com/2012/12/17/books/antifragile-by-nassim-nicholas-taleb.html?pagewanted=all&_r=0">Antifragile</a>” book: complex systems are learned by doing, practice is superior to thinking – and this obviously applies to architecture. Because it emphasizes practice and respect for people, lean management is geared to address the complex challenges of our environment.</span><span style="font-size: 9pt;"><o:p></o:p></span></span></div></div>

« The Lean Architect » - Do we need Abstraction and Planning « on the Gemba » ?

Today’s post is a first attempt at defining the role of a software architect in a “ Lean Software Factory ”, that is, an agile development o...

Read more »

Lean, Scrum, Agile and Extreme Programming

<div dir="ltr" style="text-align: left;" trbidi="on"><br /><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">Two thousand years ago, Epictetus told us “<b>i</b></span><b><span lang="EN-US" style="background: white; color: #222222; mso-ansi-language: EN-US; mso-bidi-font-family: Arial;">t is impossible for a<span class="apple-converted-space"> </span></span></b><em><b><span lang="EN-US" style="background-color: white;">man</span></b></em><span class="apple-converted-space"><b><span lang="EN-US" style="background: white; color: #222222; mso-ansi-language: EN-US; mso-bidi-font-family: Arial;"> </span></b></span><b><span lang="EN-US" style="background: white; color: #222222; mso-ansi-language: EN-US; mso-bidi-font-family: Arial;">to<span class="apple-converted-space"> </span></span></b><em><b><span lang="EN-US" style="background-color: white;">learn what he thinks he already knows</span></b></em><span lang="EN-US">”. This principle is still one of the main reasons why it is so difficult to deploy change, when new (development) methods are concerned. I have been advocating for lean and agile methods for many years and the most common pattern I hear is: “<i>I know, I do it already</i>” :)</span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">The goal of this post is to focus on the difference between agile in general, SCRUM in particular, and lean software development. I have been speaking about lean software development for three years and I have found that the (legitimate) confusion between lean and agile does not help. I keep hearing “<i>I already know about agile, scrum … or lean</i>”. I can relate to this. I started as a research scientist in the field of programming languages and software engineering twenty-five years ago, and was there (in the US & the academic community) when agile was born, I knew personally some the founders of the </span><a href="http://agilemanifesto.org/principles.html"><span lang="EN-US">Agile Manifesto</span></a><span lang="EN-US">. I started practicing agile and extreme programming when I returned to France to create the </span><a href="http://e-lab.bouygues.com/"><span lang="EN-US">e-Lab</span></a><span lang="EN-US">. I brewed my own agile philosophy for a small team of “PhD programmers” under the designation of </span><a href="http://organisationarchitecture.blogspot.fr/2011/12/grammaire-de-linaction-et-programmation.html"><span lang="EN-US">Fractal Programming</span></a><span lang="EN-US">. Since then, I have joined larger organizations and tried to promote agile methods. At first, I was promoting what I once knew and tried, but I have quickly discovered that the agile movement is growing constantly, and that we know much more today than what you learned yesterday. Hence the Epitectus quotation.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US"><br /></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">The topic of mapping the similarities and differences between lean development and agile is quite interesting because the influences and relationships make for an intricate pattern. Agile methods and its various subsequent developments have been implementing “lean principles” for many years. Lean Software Development, on the other hand, inherits from the knowledge built by agile communities. There is already a number of interesting post on this topic. </span><a href="http://www.agileweboperations.com/lean-agile-devops-related"><span lang="EN-US">Matthias Marschall’s contribution</span></a><span lang="EN-US"> is a great synthesis which is very similar to what I’ll write in a few minutes. He shows the strong relationships and the differences, and point out at the development process dimension: <i>lean</i> helps to get a larger perspective. His </span><a href="http://www.agileweboperations.com/lean-agile-devops-related"><span lang="EN-US">slideshare</span></a><span lang="EN-US"> is also interesting, but is not structured enough from my “conceptual” perspective, which I why I found useful to write my own analysis down. Another </span><a href="http://thehypertextual.com/2012/10/16/agile-et-hyperlocal-atbdx/"><span lang="EN-US">great contribution by Cecil Dijoux</span></a><span lang="EN-US"> points out the influences but stresses the benefits of lean as providing a larger perspective, especially </span><a href="http://thehypertextual.com/2012/04/25/lean-as-the-solution-to-scale-agile-methodologies/"><span lang="EN-US">the ability to scale up agile methods</span></a><span lang="EN-US"> to enterprise management and transformation. Cecil talks about </span><a href="http://www.morisseauconsulting.com/index.php/livre-kanban-pour-lit/"><span lang="EN-US">Kanban</span></a><span lang="EN-US">, to emphasize that it goes further than the traditional visual management practices of SCRUM. There are so many other examples. For instance, talks about agile methods such as </span><a href="http://fr.slideshare.net/LaurentSarrazin/2012-0525agile-francesweetrupture03final"><span lang="EN-US">Laurent Sarrazin</span></a><span lang="EN-US">’s which makes the influence of lean principles obvious. Laurent’s is a great example of a modern talk about agile, which goes much deeper that what I was hearing 15 years ago. In a reciprocate manner, great pieces about lean development, such as </span><a href="http://welovelean.com/write-up-10-things-ive-learnt-about-lean-star"><span lang="EN-US">Welovelean</span></a><span lang="EN-US">’s, are still very close to the founding principles of the Agile Manifesto.<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US"><br /></span></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">The claim of this post is : <b>there is more to lean software development than a great synthesis of the best agile methods</b>. To make my point, I will briefly summarize the key ideas of the successive steps of the evolution path from </span><a href="http://en.wikipedia.org/wiki/Agile_software_development"><span lang="EN-US">Agile</span></a><span lang="EN-US">, </span><a href="http://en.wikipedia.org/wiki/Extreme_programming"><span lang="EN-US">XP</span></a><span lang="EN-US">, </span><a href="http://en.wikipedia.org/wiki/Scrum_(development)"><span lang="EN-US">SCRUM</span></a><span lang="EN-US"> to </span><a href="http://en.wikipedia.org/wiki/Lean_software_development"><span lang="EN-US">Lean Software Development</span></a><span lang="EN-US">. I need to stress out that the following description is both too brief and incomplete. It is quite succinct because (a) I have already talked about this in </span><a href="http://informationsystemsbiology.blogspot.fr/2011/11/lean-it-devops-and-cloud-programming.html"><span lang="EN-US">previous posts</span></a><span lang="EN-US"> (b) it is easy to find better references for Agile (</span><a href="http://agilemanifesto.org/principles.html"><span lang="EN-US">manifesto</span></a><span lang="EN-US">), </span><a href="http://www.extremeprogramming.org/"><span lang="EN-US">XP</span></a><span lang="EN-US">, </span><a href="http://www.scrum.org/Resources/What-is-Scrum"><span lang="EN-US">SCRUM</span></a><span lang="EN-US"> or </span><a href="http://fr.slideshare.net/jpvajda/lean-software-development-principles"><span lang="EN-US">Lean Software</span></a><span lang="EN-US">. It is incomplete by construction, since to promote clarity, I have only selected three additional ideas/principles when moving from Agile to XP, then to SCRUM, then to LSF.<o:p></o:p></span></span></div><div class="MsoNormal"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Let me then characterize the essence of agile (when it started) as the combination of: <o:p></o:p></span></span></div><div class="MsoNormal"></div><ol style="text-align: left;"><li><a href="http://blog.extremeplanner.com/2006/08/small-teams-make-better-software.html" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Small teams</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : a strong emphasis on strong interactions.</span></li><li><a href="http://fr.slideshare.net/SteveRogalsky/agile101-small-batches" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Small batches</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : break a larger project into manageable chunks (sprints), do them one at a time and constantly re-evaluate what the next chunk will be.</span></li><li><a href="http://agile.dzone.com/articles/qa-agile-approach-0" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Time boxing</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : there is nothing like a time constraint to help you focus on what really matters. Deliver less, but deliver on time.</span></li><li><a href="http://www.agilemodeling.com/essays/executableSpecifications.htm" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Coevolution of code/design/architecture</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : the end of the waterfall model, the code and its <a href="http://www.leanessays.com/2011/08/dont-separate-design-from.html">requirements</a>are built continuously.</span></li><li><a href="http://heratech.wordpress.com/2009/12/28/agile-tenet-6%E2%80%93-face-to-face-communication/" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Role of face-to-face communication</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : A key insight of agile is that to break the waterfall model, you need to reintroduce face-to-face interaction and banish the email push, which breaks the responsibility and engagement chains.</span></li><li><a href="http://www.agilemodeling.com/artifacts/userStory.htm" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">User stories</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"> : the only way to capture a need is to give it some flesh and make it into a story. Read Daniel Pink’s “</span><a href="http://en.wikipedia.org/wiki/A_Whole_New_Mind" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">A Whole New Mind</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">” to understand why.</span></li></ol><br /> <div class="MsoListParagraphCxSpLast" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">There is a lot of </span><i style="font-family: Verdana, sans-serif;">lean</i><span style="font-family: Verdana, sans-serif;"> in these principles & practices. Obviously once cannot reduce agility to this short list of principles, I should also explain a lot of associated practices (supported with tools) regarding the development process. Let me now turn to Extreme Programming, which I will characterize with three additional ideas (or increased emphasis):</span></div><div class="MsoListParagraphCxSpLast" style="text-align: justify;"></div><ol><li><a href="http://en.wikipedia.org/wiki/Test-driven_development" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"><span lang="EN-US">Test-driven development</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;">: the best way to develop code is to write test cases first.</span></li><li><a href="http://www.extremeprogramming.org/rules/overtime.html" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"><span lang="EN-US">Sustainable pace</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;">: the only way to satisfy a customer in the software world is to last, because software is a “living thing” that evolves constantly. Hence, the rhythm must be set to reach long-term goals and respect everyone in the development process.</span></li><li><a href="http://elegantcode.com/2011/01/21/book-review-extreme-programming-explained/" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Code is valuable</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">: XP was the first method to significantly raise the status of code, through practices such as code reviews, refactoring or pair programming and principles such as “elegant code”, “do it right the first time”, etc.</span></li></ol><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US">Here, as well as in the following section on SCRUM, lean is everywhere </span><span lang="EN-US">J</span><span lang="EN-US"> That is, we can see these principles as the application of lean principles to software development, derived from the <a href="http://en.wikipedia.org/wiki/Toyota_Production_System">TPS</a>. I would then characterize SCRUM with the following additional contributions:<o:p></o:p></span></span></div><div class="MsoNormal" style="text-align: justify;"></div><ol><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"><span style="font-size: 7pt;"> </span></span><a href="http://www.xqa.com.ar/visualmanagement/" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;"><span lang="EN-US">Visual Management</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-align: left; text-indent: -18pt;">: give the team a project room, use the walls to share release notes, the backlog, planning information, etc.</span></li><li><a href="http://en.wikipedia.org/wiki/Stand-up_meeting" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Practices and Rites</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">: SCRUM is practical method that makes agile easier to learn through its rites, such as standup meetings.</span></li><li><a href="http://zacgery.blogspot.fr/2012/11/reflections-in-scrums-silver-mirror.html" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Reflection</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">: SCRUM proposes </span><a href="http://www.mountaingoatsoftware.com/scrum/sprint-retrospective/" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">sprint  retrospectives</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">, a great tool to promote continuous improvement. One could say that standing back is part of agile, but SCRUM brings a true emphasis on “doing it for real”.</span></li></ol><div class="MsoNormal"><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">These twelve sets of practices make a maturity model for SCRUM. To qualify to lean software development, I would add another three steps:<o:p></o:p></span></span></div><div class="MsoNormal"></div><ol style="text-align: left;"><li><a href="http://en.wikipedia.org/wiki/Kanban_(development)" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><span lang="EN-US">Kanban</span></a><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">: the application of visual management to the visualization of process flow and work in process (WIP). Kanban is a critical tool to reduce WIP and to implement pull (just in time), two key lean principles.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><a href="http://en.wikipedia.org/wiki/Kaizen">Kaizen</a> :  the heart of the lean approach towards problem solving. Lean makes problem solving a team activity and a learning tool. There are many associated tools and practices such as the “<a href="http://en.wikipedia.org/wiki/5_Whys">Five whys</a>” or <a href="http://sloanreview.mit.edu/the-magazine/2009-summer/50408/toyotas-secret-the-a3-report/">Toyota’s A3</a>.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;"><a href="http://en.wikipedia.org/wiki/5S_(methodology)">5S</a> and <a href="http://www.allaboutagile.com/lean-principles-1-eliminate-waste/">waste removal</a> : sort, clean-up, organize, focus on simplicity and writing less code. Some of it is part of XP, <a href="http://leansoftwarepractices.com/">but lean goes further</a>, with more tools and deeper practices (e.g. <a href="http://en.wikipedia.org/wiki/Value_stream_mapping">value-stream mapping</a>).</span></li></ol><br /> <div class="MsoListParagraphCxSpMiddle" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;">This is a simplified view of lean software (although the combined practices of the 15 previous bullet points represents a fair amount of commitment), see the seven principles of </span><a href="http://www.poppendieck.com/" style="font-family: Verdana, sans-serif;">Mary and Tom Poppendieck</a><span style="font-family: Verdana, sans-serif;"> or my own description of </span><a href="http://informationsystemsbiology.blogspot.fr/2012/07/lean-software-factory.html" style="font-family: Verdana, sans-serif;">Lean Software Factory</a><span style="font-family: Verdana, sans-serif;"> for more details. A great practical introduction to lean software development is “</span><a href="http://www.crisp.se/file-uploads/Lean-from-the-trenches.pdf" style="font-family: Verdana, sans-serif;">Lean from the Trenches</a><span style="font-family: Verdana, sans-serif;">” by Henrik </span>Kniberg<span style="font-family: Verdana, sans-serif;">.</span></div><div class="MsoListParagraphCxSpMiddle" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><br /></span></div><div class="MsoNormal" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><b><span lang="EN-US">Another key evolution of the past 20 years is the scope of agile/lean development method, which has grown from software development to product innovation</span></b><span lang="EN-US">. In the spirit of keeping this post simple, I will consider the scope with four successively embedded steps (from the smallest to the largest):<o:p></o:p></span></span></div><div class="MsoNormal"></div><ul style="text-align: left;"><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">Software development is the original scope of agile methods. It obviously includes testing, as well as the proper code management & configuration tools to make coding as efficient as possible. See my reference to Curt Hibbs in my <a href="http://informationsystemsbiology.blogspot.fr/2012/07/lean-software-factory.html">previous post on software factories</a>.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">The next step is to move to <a href="http://en.wikipedia.org/wiki/Continuous_integration">continuous integration</a>. The ability to generate a “fresh product” every night (for instance) that can be fully tested, with as many automated test as possible, yields a completely different culture and level of innovation. This requires a significant amount of tools and training. This is the case where you need to change the tools in order to change the culture.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">The third step is <a href="http://radar.oreilly.com/2009/03/continuous-deployment-5-eas.html">continuous deployment</a>, which is best illustrated by <a href="http://en.wikipedia.org/wiki/DevOps">Devops</a>. I have mentioned Devops <a href="http://informationsystemsbiology.blogspot.fr/2011/11/lean-it-devops-and-cloud-programming.html">already</a>, the big step is to include operations in the lean/agile process. Moving from CI to CD requires more tools, more automation … and a big culture change since it require to move from “agile in the lab” to “agile in the field”.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; text-indent: -18pt;">The last step is <a href="http://asqcincinnati.org/Library/continuousimprovement/ImprovementToInnovation.pdf">continuous product improvement and innovation</a>, which is beautifully explained in <a href="http://organisationarchitecture.blogspot.fr/2011/08/lean-startup-et-lean-product.html">The Lean Startup</a> from Eric Ries. In addition to continuous deployment, we need to gather continuous feedback (e.g., with <a href="http://en.wikipedia.org/wiki/A/B_testing">A/B testing</a>) and re-inject this knowledge into the development cycle. A great source to better understand lean startup principles is <a href="http://www.ashmaurya.com/">Ash Maurya’s blog</a>, and these principles do not apply to startups only!</span></li></ul><br /> <div class="MsoNormal" style="text-align: justify;"><b><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">Lean Software Development is the combination of both dimensions: the depth of lean practices and the scope</span></span></b><span lang="EN-US"><span style="font-family: Verdana, sans-serif;">. I could easily derive a maturity index from this: the maturity level would be the product of:  (a) how many of the 15 principles are really in action, times (b) what is the scope of the lean/agile process. I won’t go into this direction because lean software is much more than a development method; it is a work philosophy, in the “<a href="http://en.wikipedia.org/wiki/The_Toyota_Way">Toyota Way</a>” sense. It is mostly a work philosophy that is built on the <a href="http://organisationarchitecture.blogspot.fr/2012/11/motivation-lean-et-stress.html">motivation and satisfaction</a> of the software producers. A key insight from “<a href="http://www.amazon.fr/Implementing-Lean-Software-Development-Concept/dp/0321437381">Implementing Lean Software Development</a>” is that one must move away from a project-oriented culture based on contracts towards a product-oriented culture based on collaboration. A wonderful illustration of the scope of the lean transformation is the great book “<a href="http://www.amazon.fr/Les-G%C3%A9ants-Web-Pratiques-Architecture/dp/2952589534/ref=sr_1_1?ie=UTF8&qid=1354347141&sr=8-1">Les Géants du Web</a>”, just released by <a href="http://blog.octo.com/tag/les-geants-du-web/">Octo</a>. Its subtitle is “culture / practice / architecture”; most of what I have just described is beautifully explained and illustrated. There is lean and agile everywhere in that book, but this is a book for everyone, not reserved to managers or software developers. The book is about what makes the best “Web companies” successful, from a culture, practice and architecture point of view. However, the similarity with the lean philosophy is striking.</span><o:p></o:p></span></div></div>

Lean, Scrum, Agile and Extreme Programming

Two thousand years ago, Epictetus told us “ i t is impossible for a   man   to   learn what he thinks he already knows ”. This principle is ...

Read more »

Systemic Simulation of Smart Grids : First Results

<div dir="ltr" style="text-align: left;" trbidi="on"><w:sdt contentlocked="t" id="89512093" sdtgroup="t"><span style="font-family: "Calibri","sans-serif"; font-size: 1.0pt; mso-ansi-language: FR; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: FR; mso-fareast-theme-font: minor-fareast; mso-hansi-theme-font: minor-latin;"><w:sdtpr></w:sdtpr><w:sdt docpart="60B50BC023144821ABA7290EC34FE67B" id="89512082" storeitemid="X_66CC8B3F-D460-4797-8657-E0B3B1ECE80D" text="t" title="Titre du message" xpath="/ns0:BlogPostInfo/ns0:PostTitle"></w:sdt></span> </w:sdt><br /><div class="Publishwithline"><br /></div><div style="border-bottom: solid #4F81BD 1.0pt; border: none; mso-border-bottom-themecolor: accent1; mso-element: para-border-div; padding: 0cm 0cm 2.0pt 0cm;"> <div class="underline"><br /></div></div><div class="PadderBetweenControlandBody" style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span style="font-size: 10pt; line-height: 12pt; text-indent: 14.2pt;">The month of August provides me with the opportunity to get back to my computer and to resume my programming projects. I have been able to complete the first step of S3G (</span><a href="http://informationsystemsbiology.blogspot.fr/2011/09/systemic-simulation-of-smart-grids.html" style="font-size: 10pt; line-height: 12pt; text-indent: 14.2pt;">Systemic Simulation of Smart Grids</a><span style="font-size: 10pt; line-height: 12pt; text-indent: 14.2pt;">) which I started last year. I had the opportunity to attend the </span><a href="http://www.naefrontiers.org/Symposia/EU-USFOE/32025/2011EU-USFOE.aspx" style="font-size: 10pt; line-height: 12pt; text-indent: 14.2pt;">EU-US Frontiers of Engineering Symposium</a><span style="font-size: 10pt; line-height: 12pt; text-indent: 14.2pt;">, where I have presented my project and received encouraging and interesting feedback.</span></span></div><div class="MHeading2" style="line-height: 12pt; margin: 0cm 0cm 0.0001pt; text-align: justify;"><span lang="EN-US" style="font-size: 10pt;"><span style="font-family: Verdana, sans-serif;">The objective of S3G is to simulate the production and consumption of electricity throughout a long period of time (15 years). The S3G model, which is illustrated by the following figure, may be summarized with five parts:<o:p></o:p></span></span></div><div class="MHeading2" style="margin: 0cm 0cm 0.0001pt;"></div><ul style="text-align: left;"><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">Energy demand: for each city, energy demand is generated from an hour-by-hour and day-by-day template, adding some random variation (the extent of which is a model parameter) together with a city-specific variation. This number is then reduced by the amount of “negaWatts”, computed from the total amount invested by the city. The model uses a ratio obtained from a concave-increasing function of the investment.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">Dynamic Pricing: both suppliers and operators use a simple affine pricing model, with a constant price when the demand is less than a “base power”, and a linear formula when the demand is higher.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">Production: suppliers use nuclear power according to planned schedule and adjust to resulting demand with fossil plants. Operators always use their green power (store it in the “buffer” or resell it when there is too much of it). They adjust to the city demand with their own fossil plant and wholesale electricity from suppliers, at the lowest marginal cost.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;"><span style="font-size: 7pt; line-height: normal;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">Consumption: The actual electricity consumption for each city is the demand, minus “shaving”, which is obtained by applying an S-curve to the sale price.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">Market-share: for each city, the market balance between the national supplier and the local operator is determined yearly using another S-curve.</span></li></ul><br /> <div class="MHeading2" style="line-height: 12.0pt; margin-bottom: .0001pt; margin: 0cm;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyXOwvxedvl-ED4CIdQZNX6S3gAieOSMNzlkQhG6iRuWOCVs_cL9nDdUg1z0DzACJZizEmAPi8HgKQ_hPMLaeuqAt9ndyqo_MfABSmk2fTOHmBXwtEEkUQPaEpeit9gQDquQrEPeU3cd4/s1600/Diapositive1.JPG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyXOwvxedvl-ED4CIdQZNX6S3gAieOSMNzlkQhG6iRuWOCVs_cL9nDdUg1z0DzACJZizEmAPi8HgKQ_hPMLaeuqAt9ndyqo_MfABSmk2fTOHmBXwtEEkUQPaEpeit9gQDquQrEPeU3cd4/s640/Diapositive1.JPG" width="640" /></a><span style="font-family: Verdana, sans-serif;"></span></div><div style="text-align: justify;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="font-size: 10pt; line-height: 12pt;">This S3G model is both simple and complex. It is simple because it is based on a handful of equations, resulting in a simulation code that is 500 lines long. On the other hand, it is a complex model for two reasons. On the one hand, there are multiple feedback and interaction loops that make it difficult to analyze how the system will react to perturbations. On the other hand, there are many unknown parameters in this model (such as market sensitivity, demand-response behavior, negaWatt capabilities, etc.).</span><span lang="EN-US" style="font-size: 10pt; line-height: 12pt;"> </span></span></div><br /><div class="MHeading2" style="line-height: 12.0pt; margin-bottom: .0001pt; margin: 0cm;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="font-size: 10pt;"><br /></span></span></div><div class="MHeading2" style="line-height: 12.0pt; margin-bottom: .0001pt; margin: 0cm;"><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="font-size: 10pt;"><br /></span></span></div><div class="MHeading2" style="line-height: 12.0pt; margin-bottom: .0001pt; margin: 0cm;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBur6yqBxsJcyAB2eYRNSmsgScw8FBuTsIMEBmmAKPFEaX-Y4DuVG9UtLoTf4s3wqnA7XMuqebL4kpT8tfuHnRUlBNeHif7ano-qquIdmkWWtCOO138KDHjnMybAqj0LDTZLOF1vAQd40/s1600/S3G-FoE-Poster-Dec12.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBur6yqBxsJcyAB2eYRNSmsgScw8FBuTsIMEBmmAKPFEaX-Y4DuVG9UtLoTf4s3wqnA7XMuqebL4kpT8tfuHnRUlBNeHif7ano-qquIdmkWWtCOO138KDHjnMybAqj0LDTZLOF1vAQd40/s640/S3G-FoE-Poster-Dec12.jpg" width="640" /></a><span style="font-family: Verdana, sans-serif;"><span lang="EN-US" style="font-size: 10pt;">The following figure shows, on the right part, a rough summary of the simulation loop that is run for each time period (3 hours, hence 8 times per day). On the left part, it shows the three main <a href="https://sites.google.com/site/yvesresearchagenda/home">GTES generic procedures</a>.<o:p></o:p></span></span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQoHEooBvOHZnz7dU7e-xTRPQvxwC5RQUOTf9jk3kji9Oo9pR-enFz9kSUFeYuOvkN7wRSubPBlQAo8WClqhQ45GSkFev26fHMSvXZdSCOMwQfj6FQsuhQO4VnpJdCzLF0RTCyBd-qLJ0/s1600/S3G-simulation-loop.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQoHEooBvOHZnz7dU7e-xTRPQvxwC5RQUOTf9jk3kji9Oo9pR-enFz9kSUFeYuOvkN7wRSubPBlQAo8WClqhQ45GSkFev26fHMSvXZdSCOMwQfj6FQsuhQO4VnpJdCzLF0RTCyBd-qLJ0/s640/S3G-simulation-loop.jpg" width="640" /></a></div><div class="MText" style="line-height: 12pt; text-align: justify;"><span style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt;">I have already spent some time on computational experiments that will be presented at </span><a href="http://www.csdm2012.csdm.fr/" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt;">CSDM</a><span style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt;">in December. </span><span style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt;">A GTES simulation run returns the average and standard deviation of a few key business parameters, as well as some indication of the </span><a href="http://organisationarchitecture.blogspot.fr/2008/01/lapproche-gtes-et-les-quilibres-entre.html" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt;">Nash convergence</a><span style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt;">. Giving averages and a few deviations is a poor restitution of the rich data gathered during the computational experiment, but the goal is simply to “get a feeling for what is happening”, as opposed to producing a forecast. The current set of experiments is designed to understand the main issues that were exposed in </span><a href="http://informationsystemsbiology.blogspot.fr/2011/09/systemic-simulation-of-smart-grids.html" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt;">the previous post</a><span style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt;">. I have defined a fictional country somehow similar to France decomposed into 10 regions/cities. I have run 8 experiments that may be defined as follows:</span></div><div class="MText"></div><ol style="text-align: left;"><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;">The “default” is a reference point, from which “what-if” sensitivity analysis is made. The economic parameters are set in such a way that alternate operators start with a 20% market-share and should be able to increase it if they demonstrate a better management of variability.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;">The second experience raises the variability of energy consumption (globally), while the third experience raises the local variability (each city is more different from each other).</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;">The fourth experiment doubles the fossil energy price (gas and coal). In the default scenario, it is randomly drawn between 20€ and 40€/MWh.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;">The fifth experiment imposes a 5% reduction of the nuclear assets for the supplier during the first 5 years.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;">The sixth experiment sets a carbon tax at 100€/t, the proceeds of which is used by the “regulator” to subsidize green energy investment.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;">The seventh experiment explores the impact of overall demand variation in the next 15 years. The model assumes a constant growth/decline of electricity demand which is expressed as a percentage. This experiment plays with different values to see if <i>demand impacts the profitability of smart grids</i>.</span></li><li><span style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;">The last experiment is a small variation of the first one, where wholesale prices are more rigidly constrained. During this set of experiment, I considered that the constraints that regulate wholesale prices for the supplier are fixed. An interesting next step will be </span><i style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;">to make them a strategic parameter for the regulator</i><span style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;"> (which is a better representation of reality and makes for a more interesting game).</span></li></ol><div style="text-indent: -23.799999237060547px;"><span style="font-family: Verdana, sans-serif; font-size: x-small;"><span style="line-height: 16px;"><br /></span></span></div><span style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt;"><div style="text-align: justify;"><span style="font-size: 10pt; line-height: 12pt;">I still need to do some additional tuning before presenting my results at CSDM. I will then make my slides available on this blog. For once, I </span><i style="font-size: 10pt; line-height: 12pt;">need to run longer simulations</i><span style="font-size: 10pt; line-height: 12pt;"> (with larger samples) to ensure that the Monte-Carlo sampling is stable. I also want </span><i style="font-size: 10pt; line-height: 12pt;">to introduce better local optimization meta-heuristics</i><span style="font-size: 10pt; line-height: 12pt;">, such as Tabu search. One of the key issues for smart grid is the pricing model, both for the operator and the supplier. The more complex the pricing model, the better the actor may play with the </span><a href="http://en.wikipedia.org/wiki/Demand_response" style="font-size: 10pt; line-height: 12pt;">demand-response</a><span style="font-size: 10pt; line-height: 12pt;">concept. However, a more complex princing model requires better optimization techniques to ensure that the automatic computation from GTES (where each actor “learns” its “best response”) is relevant.</span></div></span><br /> <div class="MText" style="line-height: 12.0pt;"><br /></div><div class="MText" style="line-height: 12.0pt; text-indent: 0cm;"><span lang="EN-US" style="font-size: 10.0pt; mso-fareast-language: EN-US;"><span style="font-family: Verdana, sans-serif;">This being said, here are a few findings that may be drawn from the hundreds of runs made with the S3G model, and that are worth sharing: <o:p></o:p></span></span></div><div class="MText" style="text-indent: 0cm;"></div><ul style="text-align: left;"><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;"><span style="font-size: 7pt; line-height: normal;"> </span></span><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -17.85pt;">There is a systemic benefit of distribution and autonomy to cope with variation. This is shown in our second and third line. It is a “subtle” variation (small effects), which means that the economy of the local operator is dominated by its capacity to operate at much lower customer management costs than the supplier.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">CO2 tax increases play a very small role, and one that is difficult to anticipate since it both favors the local operator (support green subsidies) and the supplier (raises the difference between fossil and nuclear).</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">“De-nuclearization” is a favorable scenario for smart grid operators, as are most regulations that are adverse to the supplier. The obvious limitation is the resulting price increase that reduces the total economy output (and the country’s competitiveness).</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">The “community advantage” (that is, the ability for a local operator to better manage the demand-response loop because it is “closer” to its end customer) is marginal, and it is quite unclear if the payback from demand-response management is enough to sustain the operator’s business model.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">Investing in local storage is never an interesting option (at current prices). We needed to slash the price by over an order of magnitude to see a viable payback in less than 10 years.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">There is a clear competition between local operators and suppliers. The learning component of GTES makes for “agile” players who react closely to each other signals. The pricing structure plays an important role (we have only explored a simple variable pricing scheme). A logical consequence is the importance of regulation.</span></li><li><span lang="EN-US" style="font-family: Verdana, sans-serif; font-size: 10pt; line-height: 12pt; text-indent: -18pt;">The results are sensitive to the strategies of the player. <i>A next step for S3G is to build a “strategy matrix”</i>, which is a tabular “what-if” sensitivity analysis where we see what happens if the goals of the players are changed.</span></li></ul><br /> <div class="MsoNormal"><br /></div></div>

Systemic Simulation of Smart Grids : First Results

The month of August provides me with the opportunity to get back to my computer and to resume my programming projects. I have been able t...

Read more »