information system biologis

Flexibility and Biology

In a previous post I have listed flexibility as one of the four pillars of agility. I defined flexibility very loosely with the following sentence: " the technical ability to cover a large span of business situations and processes with the existing set of IT services and capabilities." 			This requires a little bit of further explaining. Especially, there are two ways to obtain this ability which are both relevant: <ul><li>The ability to implement change in order to achieve functional flexibility. This is foremost a matter of architecture. Hence it is the first thing that comes to mind, in the context of "SOA and Agility". This flexibility is expressed at the component level (the ability to parameterize: change the functional behavior though a change of parameters –cf. the 12th chapter of my book on SOA), and at the "integration level", i.e. the ability to integrate a new component quickly and efficiently. </li><li>The ability to adapt to a new situation "without change", that is without having to stop and modify one or more components. This kind of flexibility is a combination of "meta-data architecture", that is, the ability to change the behavior of information systems through the change of parameters and the ability to do so dynamically. This is the most important part, and the one that is most often overlooked. In many cases, changing parameters is still a new project that requires testing, synchronization and going live with a new version. </li></ul>We can summarize by defining flexibility as the combination of functional flexibility (ease of change) and operational flexibility ( at run-time, without change to the existing binaries). The first kind may be labeled "mechanical flexibility". The analogy makes sense because when you modify a mechanical device, you must (most often) shut it down before changing or adding a new piece. Most of what you can read about IT agility is actually of this kind. New technology, rule-based engines, meta-data parsers are all designed to extend the functional flexibility. This is definitely true of integration infrastructure technology that I have come across during the last few years. There are a few noticeable exceptions (such as using UDDI for dynamic discovery of services) but … surprisingly … they have not made it to mainstream IT yet. I will call the second kind "organic flexibility" for obvious reasons: changing the SI without shutting it down (without the "go live" step) reminds us of living organisms which grow, change and adapt themselves without having to halt their biological processes. The boundary is subtle: everyday running systems are modified by the input of their users. So what? It turns out that for the same systems (at least those that I known of) there also exists "hidden"/admin parameters that cannot be changed without stopping and restarting the system. It also turns out that most major changes require writing new code, to re-compile, re-link or re-connect, hence a new "go live". I must confess that I have been trained for twenty years to think about flexibility in a mechanical way. However, I now believe that the flexibility that matters the most is the organic kind. This will become truer and truer as the stringent requirements of real-time digital business will demand no shut-downs of any kind. Organic flexibility is a matter of organization, people and processes as much as of technology. However, there is a technological part: one may not achieve this behavior (easily) with legacy technology. A key is obviously the availability of dynamic meta-data that is interpreted as parameters and modifies the behavior in a deep way. The challenge starts when different components must share a change of parameters to achieve a new behavior. This means and requires that propagating new parameters becomes a IT run-time service. The technology for this type of propagation is there, once again we find rule-based engines, scripting engine, automation tools such as processflows. If I reflect on the IT projects that I have been a part of, 80% of the energy is spent making sure that there is enough flexibility ("enough parameters") to make the component "future-proof", and 20% is spent thinking about the processes that are used to change these parameters. Designing an organic information system requires to change the ratio to the opposite.

Flexibility and Biology

In a previous post I have listed flexibility as one of the four pillars of agility. I defined flexibility very loosely with the following se...

11:31 AM

Software Development Productivity Gains ?

As I am close to getting my last book published, I figured I should publish a few pages once in a while to trigger the interest and to gather a few comments. <div align="justify"> This first extract deals with the matter of productivity gains in the information system. This is a complex topic, and it is really the whole of this book that provides an answer to it. We will merely highlight the most important points and indicate the chapters where they are described in greater detail. In this initial overview we will distinguish the project aspect, dealt with in this section, from the operations aspect, dealt with in the following section. These can then be broken down into three major parts: technology, industry and architecture. Since the dawn of IT, every year has seen the advent of new technologies that promise spectacular gains in productivity<a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftn1" name="_ftnref1" style="mso-footnote-id: ftn1" title="">[1]</a>. In the past, people talked about programming languages with a high level of abstraction, artificial intelligence, computer assisted graphic design, synthesis and automatic program checks, etc. Today, the hot topics in the literature consist of meta-programming, service architecture, component assembly, to name but a few. We will go back over these technologies and their promises in chapter 9. There is certainly progress, in terms of function points per dollar and in terms of function points per man-day, but this progress is coming at a very slow rate. If we had to provide an order of magnitude based on the comparative analysis of the productivity figures from the late 80s, and those that are found today, I would say that we have gained a factor 2 in 20 years.<a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftn2" name="_ftnref2" style="mso-footnote-id: ftn2" title="">[2]</a>. It is a matter of overall productivity (on the project perimeter) according to the technical axis (in FP/man-day). The economic aspect is different, because the IT industry has undergone significant changes.</div><div align="justify"> The software and system integration industry brings with it structural gains in productivity according to the following factors:</div><ul><li><div align="justify">Globalization, which makes it possible to move a part of production into countries where there is a lower cost of labor. This delocalization produces a substantial economic advantage, when it is relevant, particularly for stable needs. We shall come back to this in chapter 9.</div></li><li><div align="justify">Mutualization of needs, which leads to the continuous release of software packages that capitalize on the needs shared by different companies. The associated gains may be quite sizeable at the level of a project, but are only moderate at the level of the average of the ISD.</div></li><li><div align="justify">Maturing and professionalization of development practices in relation to the development of tools. The continued formalization and optimization of development processes by using standardized repositories as support<a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftn3" name="_ftnref3" style="mso-footnote-id: ftn3" title="">[3]</a>, is a good practice that is vital for an ISD and its providers. </div></li></ul> These productivity factors may have greater effects than technology factors, but their implementation is not easy, and requires discernment in terms of the scope of application. The third area of progress is the information system architecture, which we distinguish from the technology section, because it involves what is known as “enterprise architecture”, a subject which spans information technology, business processes and “strategic alignment” of the information system. Enterprise architecture is used as a complexity control strategy so that the integration costs of a new application do not reach an irreversible level as the IS grows. This is mostly a defensive strategy that is used when the complexity is precisely such that integration becomes the predominant part of IT projects. As a result, the effect that is noticed is not a “productivity miracle” but the return to a “comfort zone” where integration costs are proportional to the functional complexity of the application. The conclusion we can make at this stage of the analysis is that there are undisputed productivity factors in terms of information system development but these are not necessarily easily deployed; the gains remain measured. The visible part of these gains is all the more modest that another part is consumed by three heavy trends of enterprise IT<a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftn4" name="_ftnref4" style="mso-footnote-id: ftn4" title="">[4]</a>: <ul><li><div align="justify">Additional requirements, as regards availability, security, ergonomics, etc., which induces more complex IT solutions with an equivalent functional perimeter.</div></li><li><div align="justify">Inter-operability requirements, whether within the company or in conjunction with partner companies. Enterprise IT has been experiencing a double revolution for 20 years. Firstly, its functional scope grew along with inter-application connectivity requirements (unlike departmental IT in the 70s and 80s). Secondly, the concept of the extended company arose, i.e. the necessity to connect the information systems of companies that are collaborating on processes.</div></li><li><div align="justify">The abstraction of software functions to control complexity. Controlling complexity leads to the creation of layered software, with the layers corresponding to different levels of abstraction, and makes it possible to have them evolve without having to get a sense of them as a whole<a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftn5" name="_ftnref5" style="mso-footnote-id: ftn5" title="">[5]</a>. These methods make it possible to control application systems that are very rich and complex but generate additional costs (the actual size exceeds the expected size). </div></li></ul> Users of general-public software such as office suite tools will have noticed that more and more resources are required to run their favorite applications (e.g. word processing), mostly due to these three reasons.<div align="justify"> <a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftnref1" name="_ftn1" style="mso-footnote-id: ftn1" title="">[1]</a> We have already referred to F. Brooks’ article entitled “No Silver Bullet: Essence and Accidents of Software Engineering”, which examines the intrinsic difficulties in terms of software productivity. This article can be found in T. DeMarco and T. Lister’s Software State-of-the-art, which includes a selection of the best articles from the late 80s. <a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftnref2" name="_ftn2" style="mso-footnote-id: ftn2" title="">[2]</a> This assessment is an average ratio which hides a great level of variability, according to the type of problems to be dealt with. This will become clearer when inter-company benchmark test results are published every year. <a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftnref3" name="_ftn3" style="mso-footnote-id: ftn3" title="">[3]</a> The repository which is gradually becoming the standard one in IT project development is CMMI (Capability Maturity Model Integration). For more on CMMI, see the first section of M.B. Chrissis, M. Konrad and S. Shrum’s CMMI – Guidelines for Process Integration and Product Improvement. You can also find many sources online, including those of the SEI (Software Engineering Institute). <a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftnref4" name="_ftn4" style="mso-footnote-id: ftn4" title="">[4]</a> This phenomenon is more commonly known in technology circles as the “rebound effect”. For example, the efficiency increase for heating a square foot of living space is offset by the increase in the average house size; the progress in car fuel consumption has helped to develop the usage, instead of simply generating savings, etc. In IT, the “complexity race” absorbs a proportion of the gains by the rebound effect. <a href="http://www.blogger.com/post-create.g?blogID=438942112364524044#_ftnref5" name="_ftn5" style="mso-footnote-id: ftn5" title="">[5]</a> The concept of “layers” is hard for non-computer specialists to grasp. It is quite relevant to compare it to geology: layers corresponding to age, with a successive deposit of “functions”. For example, I have the experience of precisely analyzing the code of a major American publisher on typical functions whose performances seemed to be disappointing. Using the debugger as an analysis tool showed several “layers” which have been built up as the software has undergone different changes (from an initial effective kernel to multiple generic interfaces). This concept of “sedimentation” will be introduced in section 2.4.1.</div>

Software Development Productivity Gains ?

As I am close to getting my last book published, I figured I should publish a few pages once in a while to trigger the interest and to gathe...

3:34 AM

Agility, Biology and SOA

I have just finished writing a new chapter for the third edition of my book « Entreprise Architecture & Business Process Management » (which is only published in French, unfortunately). This new chapter deals with "SOA and Agility", two buzzwords of these IT times. A part of this chapter deals with the concept of "sustainable architecture" following what I wrote in my previous post. I have proposed to paraphrase the definition from the <a href="http://en.wikipedia.org/wiki/Brundtland_Commission">Brundtland</a> report: "Sustainable development is development that meets the needs of the present without compromising the ability of future generations to meet their own needs." 				into a definition of that spells out the "sustainable development" of the IT architecture: "Sustainable architecture is the Entreprise Architecture that enables the development of today's IT services without compromising the ability to develop tomorrow's services because of an ever-increasing complexity or the induced scarcity of resources (money or skills)." To gain this sustainability, I have been advocating for quite some while for "fractal" architectures (hence, "fractal" methods). I mean methods that can be broken into pieces and applied differently (and at different rates) to different situations. This fractal view applies to middleware, to enterprise architecture, but also to data models or to process models. The information system (as a whole) needs to grow (as a living object) according to different rates of pressure that are applied to its different parts. The <a href="http://www.ceisar.org/describe-the-enterprise/business-process-modeling.html">CEISAR</a> architecture documents make a very elegant and clear point about the value that exists when breaking long "end-to-end" business processes into smaller, more manageable pieces. The major part of this new chapter is devoted to agility (mostly because I hear too much nonsense about solving the agility issues with technology). To give a short summary, I see four forces that contribute to agility: <ul><li>Anticipation: being agile is implementing changes rapidly enough to meet TTM (time-to-market) constraints. A smart way to think about it is to start early. It is not a "solution in itself" (since many changes may not be anticipated) but it is definitely part of the "whole package" since many "smart decisions" (the one that will increase agility) take time to get implemented. </li><li>Flexibility: the technical ability to cover a large span of business situations and processes with the existing set of IT services and capabilities. </li><li>Leanness: the process of applying changes to the information system (for instance, developing a new IT project) needs to be as lean as possible (cf. my other <a href="http://organisationarchitecture.blogspot.com/">blog</a> for more details, although I will most certainly be back on this topic). Lean obviously means short from an organizational perspective (the longer the chain between the developer and the problem owner, the less agility is observed). The ultimate goal is to see IT project disappear. The "absolutely agile" IT department is the one who is not needed to let the users adapt their information system to the business changes. Although it may sound silly, it is actually a sound goal, but one that requires a lot of anticipation. </li><li>Skills: one needs to be competent to be agile! This also sounds dumb, but actually it is a rather profound observation, which is backed by observation. The fear of failure and the pressure in today's modern organizations means that any lack of full understanding or full confidence will translate into multiple redundant checks, back-and-forth, "safety precautions", etc. Another dimension is the ability to profit from the opportunity provided with the constant progress of technologies (from new IHM techniques (Ajax, mash-up, etc.) to new middleware tools, including more radical options to leverage the value in SaaS – software as a service). </li></ul>Notice that, except for the second one, these are organizational traits and not technical ones. This, by the way, is also true of being able to leverage the value of SaaS. The software services provided by SaaS are not especially great. Using them is smart from a business and an organizational perspective, and requires "business agility/flexibility" even more than technical skills or abilities. As I wrote earlier, SOA's main contribution is to provide a framework for sharing and reusing services. It also helps to achieve agility, but more as a second-order consequence rather than a direct cause-effect path, and not without efforts and dedication. The first-order benefits of reuse and share are cost and complexity reduction. On the other hand, as I have already explained earlier, SOA is foremost a governance principle, before being an architecture framework. If the stakeholders do not play "the game", the existence of a beautiful ESB (enterprise service bus) will not provide as much benefits as expected. As a governance principle, SOA sets principles and rules that must be followed. If this sounds very different from a crude idea of what a "biology-inspired" IS could be … it is indeed! The strength of <a href="http://www.biomimetik.dk/English">biomimecry</a> as a design principle is found in what nature does very well: <ul><li>Flexibility/ adaptability (to new situations) </li><li>Reliability / Survival ability </li></ul>One thing that nature does not do so well is the "economy principle", the "simplicity of design". When biologists study mechanisms in living objects, they are often surprised by the level of redundancy, by the complexity of the processes (which have been obtained through evolution). By the way, when we look at legacy code, we often get the same impression J … There are two consequences: <ul><li>Biomimecry is not enough as a design principle. The economic reality of business requires implementing a "principle of simplicity" and a constant strive to reduce costs. I advocate for a "fractal architecture", which may sound like an oxymoron to some, as a weakened form of the necessary dictatorship of principles and rules. Rules are necessary to produce simplicity. </li><li>For all the good reasons mentioned in earlier posts, it is a good idea to borrow from Nature to design systems which are at the same time: complex, reliable, flexible (doing any of the two is OK, doing the three is a challenge). However, this will come at a cost (mostly redundancy). To justify the effort, one must look at the "big picture" and consider full "life cycles" of IT services and take multiple business scenarios into account. This means that the projects that will bring biomimecry into the IS architecture will not have a simple ROI justification, but rather a sophisticated one. This is discussed in my <a href="http://www.dunod.com/pages/ouvrages/ficheouvrage.asp?id=50596">book</a>, which should be made available in English (at last) in a few months. </li></ul>Nature is organized around a simple principle that says that evolution cannot be stopped, which also applies to information systems. Hence the only mechanism to cope with complexity is to clean, to remove unnecessary things. Nature does great job of cleaning (and recycling) unnecessary things in living organisms. This is another aspect which may be borrowed, especially when managing a SOA. A key issue is to master the complexity (and the sheer number) of services. The book will mention a few tools and techniques to manage this catalog of shared services, but the first principle is to limit its size. Since evolution cannot be stopped, old services need to be removed. This blog is moving very slowly (although one would think that there are enough provoking statements in this message to generate a few comments). I actually thought that it was dead, but there are a few readers coming everyday (that's what the stats tools say). I may accelerate the posting rhythm in a few months, when I'll be preparing a new course on the "Theory of Information System".

Agility, Biology and SOA

I have just finished writing a new chapter for the third edition of my book « Entreprise Architecture & Business Process Management » ...

6:00 AM

Sustainable Enterprise Architecture

After five years spent as a CIO, I have developed the following conviction: [1] Major architecture re-engineering projects are fuelled by pain (one could say : no pain, no gain :)). Enterprise Architecture (EA) projects (I should rather say programs, as in a family of projects) require an unusual amount of effort and alignment throughout a long period of time. The alignment is even more difficult to achieve than the sustained level of effort, especially with a large IT organization. Alignment here means the fact that a large group of people decide to make sub-optimal choices, from their own viewpoint, to achieve a larger-scope goal. It may be strange way to look at alignment but it makes sense: if the logical choice for each actor was to move towards the same direction, there would be nothing to talk about :) Introducing an EA scheme (what we French call "urbaniser" the information system) usually occurs because a blazing limitation of the IS has been found. It is too slow, not flexible or agile enough, not reliable enough and, most often, too expensive, etc. The level of pain is necessary to break through a "decision/action" threshold, since there are obvious risks. From a technology perspective, an EA relies on an integration infrastructure (ESB, EAI, ETL, and so on). From a culture perspective, new concepts and a new vocabulary are introduced. What happens if the program is successful? After a while (it could be a long while :)) the pain recedes. As a consequence the alignment starts to weaken. This is not simply an internal/organizational issue for the IT department. Deploying an EA approach is a corporate endeavour, which requires from all business division a common wish to build a global system. The alignment here means that each division is ready to relinquish some of its interests for the common good. Because of this weakening of the common resolve, the master plan becomes too heavy to carry and one returns to (some of) the previous faulty behaviors that caused the pain in the first place… I have been thinking about this for the last three years and I have come to build a second conviction: [2] Service Oriented Architecture is the sustainable approach to develop a "well-architectured" information system over time. This statement probably sounds lame and dull to anyone familiar with the SOA (Service Oriented Architecture) concept. All possible "claims to fame" have already been made when SOA is concerned :). I should first state a few caveat: <ul><li>I do not mean SOA as a technical architecture framework (Web Services, ESB, …) but as a governance method for building reusable information system assets. I do not want to dwell on this today, I'll come back to it some other time (one may look at the discussion about the <a href="http://www.column2.com/2005/11/service-oriented-business-architecture/">SOBA</a> acronym to grasp this ambiguity). </li><li>I do not mean a unique, shared, common architecture. As I explained in my first book, I am a strong believer in diversity. Actually, one of the first <a href="http://de.scientificcommons.org/16317743">papers</a> to talk about "sustainable enterprise architecture", from <a href="http://de.scientificcommons.org/marten_schoenherr">Marten Schoenherr</a> and <a href="http://de.scientificcommons.org/stephan_aier">Stephan Aier</a>, precisely considered sustainability a benefit of a distributed approach. I refer my customary readers to page 98 of my <a href="http://www.dunod.com/pages/ouvrages/ficheouvrage.asp?id=50092">book</a> where I also develop this idea (e.g., the main benefit of SOA compared to earlier approach, such as EAI, is the ability to decentralize the EA program). </li><li>I really push the analogy with "<a href="http://en.wikipedia.org/wiki/Sustainable_development">sustainable development</a>" to the core: a sustainable EA approach is one that produced benefits without requiring so many efforts from the culture, the people, the organization that it stops whenever the actors have the freedom to do so. This is really about people, and especially the relationship between business process owners and their IT providers. </li><li>I am discussing about a large-scale enterprise and its information system as a whole. I consider the problem of "SOA at a departmental scale" solved (this is illustrated by the existence of so many successful implementations …). The sustainable alignment of a medium-size information system is not such a formidable task :) </li></ul>Hopefully my second book will be available soon to English-speaking readers (since I have finished the translation). They will see that one of my central theme is that a "well designed IS" is a corporate responsibility, not something that may be left to the CIO. The CIO may take the leadership for a "special re-engineering" program/effort, but this cannot last. Eventually it is a matter of management culture (unless the CIO wants to become a "dictator" but he or she usually gets fired quickly if this temptation is too strong :)). Pierre Bonnet recently opened a web site about this very topic: <a href="http://www.sustainableitarchitecture.com/home">http://www.sustainableitarchitecture.com/home</a>. His book about the same topic will be out next month. The web site is really interesting, together with the companion site about the <a href="http://www.praxeme.org/">Praxeme</a> method. If you go and read through it (<a href="http://www.sustainableitarchitecture.com/home2">which I encourage you to do</a> :)), you may think that it develops a similar line of ideas (obviously, with more details and more thought-though principles). I actually agree with everything … but I do not think that it reflects truly what sustainable development is really about: people. I am personally a big fan of the ACMS approach (Agility Chain Management System). Unfortunately, (or fortunately, since no so many people may understand what it :)), this is not where I see the issue for deploying a SOA Enterprise Architecture in a sustainable way. There (rightfully so) a lot of talk about SOA governance nowadays. Unfortunately it remains complex and abstract, whereas the issue is the appropriation from all stakeholders in the company. I will return to this topic in further postings, since I believe that this (SOA governance) is the key to agility. I fear for those who will promise agility from the sole technical merits of a SOA architecture. It turns out that there is a totally different meaning for "sustainable IT architecture" ! If one looks at the electric consumption of a data center, it is raising dangerously over the years (with respect to the double issue of energy price increase and greenhouse gas emissions). Electric consumption here includes both the powering of the computers and their cooling. Both tend to be proportional to the square of the processor frequency (one way to look at it, although the resistance decreases with the smaller scale designs). Both tend equally to be proportional to the amount of computation that is made, which is clearly growing fast in most companies. This is why Google is seeing energy consumption as a key issue. For instance, read this newspaper <a href="http://www.tgdaily.com/content/view/21524/113/">article</a> to find out about Urs Hoelzle approach to reduce server electricity footprint (more technically-savvy readers may look at <a href="http://209.85.163.132/papers/power_provisioning.pdf">this</a> :)). Since then, Hoelzle has said Google is looking into <a href="http://www.cio.com.au/index.php/id;1119156387;fp;4;fpid;51250">neutralizing its carbon emissions</a> by the end of the year. The link with architecture is as follows. The simplest way to increase the computing power without increasing the consumption is to use massive parallelism. I don't have time to go into details today. One may look at the <a href="http://storagemojo.com/2007/08/19/benchmarking-energy-efficiency/">StorageMojo</a> web site to get a lot of interesting stuff. I have just finished Ray Kurzweil book "the Singularity is near" (2005). As usual, this is a fascinating book, especially from this "sustainable development of IT" perspective. From a general perspective, it is a refreshing view from a "technology optimist" which offers a clear break from the prophets of doom. As far as computing is concerned, Kay Rurzweil offers hope for software designers to be able to use much, much faster hardware (although, if you read the book, you'll see that the name may no longer be appropriate :)), something that I am dreaming off each time I run of my "game theory simulation" :) Ray Kurzweil's optimism does not cancel the validity of Google concerns (different time scale). One might say, then, that a sustainable architecture needs to run on a grid-like structure (or any other form of massively parallel system architecture).

Sustainable Enterprise Architecture

After five years spent as a CIO, I have developed the following conviction: [1] Major architecture re-engineering projects are fuelled by pa...

10:14 AM

Lean Information Systems

Lean Manufacturing is a powerful concept, which is often misunderstood. It was made popular by Toyota’s implementation and <a href="http://organisationarchitecture.blogspot.com/2007/09/taiichi-ohno-le-maitre-de-lefficacit.html">Taiichi Ohno’s </a>vision (one of Toyota charismatic leaders). A very simple way to explain what it is would be to compare two production shops: <ul> <li>one shop is organized so that each machine is run at optimal capacity, in its best operating conditions. Buffers are introduced and the transport between machines is a little longer (so set up the machine optimally)</li> <li>the second shop is organized so that the flow is shortened as much as possible. Buffers are reduced (and eliminated as much as possible) and the transport is optimized. The consequence is that each machine is no longer working optimally. Some are underutilized and others are working in operations mode that do not yield the best productivity.</li></ul>What does Lean Manufacturing (and experience) say ? Obviously the first shop costs less to operate (cost for producing one unit) on paper, but unless it operates in a ideal world with no variations at all, it actually costs more in real life. The second approch costs less from an inventory perspective, but mostly it is more flexible (with respect to priority changes) and more robust (with respect to load variations).Let us now consider two information systems, from our scope of large-scale, distributed information systems (many parallel nodes running business processes) :<ul> <li>The first one has been designed so that each node is running close to its optimal capacity. A node here may be a group (cluster, farm, blade) of servers that run services which are the elementary components of the business processes. The computing power of the node is designed so that the node is running at 85% capacity when the load is full (i.e. when the business processes are running at their maximal expected load).</li> <li>The second one has been designed to speed up the process running and to avoid "queuing waiting time". Hence the computing power of the node is adjusted so that the average utilization ratio is closer to 50%</li></ul> Here also, the first data center is clearly cheaper to build than the first one. The second one has a few advantages: better SLA (service level agreement) may be promised to the customer (tighter = faster garanteed response time), and the upgrade process (when the company grows) may be planned in a more regular way) ... but let us assume that these are not compelling advantages. That is, let us suppose that the customer accepts the two different SLA:<ul> <li>in the first case, the SLA is such that the target response time will be obtained in 98% of the time with regular business conditions.</li> <li>in the second case, the SLA is also such that the response time will be obtained in 98% of the time (hence, a smaller number than the first one).</li></ul>I have made some interesting computing experiments last month to see how these two data centers would behave when "a little stress occurs". Stres here may come from one node unavailability, from a process overload, or from a higher-than-usual variation in the processing load. Anyone who has any experience with operations will recognize that these are the common issues of day-to-day production life. These experience were reported in a talk that I gave at the "<a href="http://www.lix.polytechnique.fr/colloque.php">Colloque d'Automne du LIX</a>", from which I have extracted the last slide: <img alt="" border="0" id="BLOGGER_PHOTO_ID_5120310555350492914" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9dgKvCbpKvGef5JZjgUzfsNewd__aB6-R3WjYosdtxf5EbuSlwJnzXavv5YPDIey4wFrBFUDI7PLVPMb0pX_Kbgvzkz-3EWvwXC11Nr14ROHuKGyLjVfcyb-kCJ1EHFRTjTF8wPo8Q7M/s400/CALSept07.jpg" style="DISPLAY: block; MARGIN: 0px auto 10px; CURSOR: hand; TEXT-ALIGN: center" /> You may find the complete presentation on the <a href="http://www.lix.polytechnique.fr/~liberti/cal07/presentations/">CAL web site</a>. To keep things simple, the curves describe the behavior of the systems (1) and (2), with different stress scenario. The different curves correspond to different strategies of "adaptive middleware" (recall that I have this interest for autonomic computing :)). What matters here is that the lower curve reproduces the strategy that ALL existing systems use today (first-come, first served). What you may see is a tremendous difference: <ul><li>The lean IS (on the left) does actually very well under stress. Only the loss of a node creates a real problem (and it is not major, the SLA drops to 75%)</li><li>The loose IS (on the right) is definitely not robust. The stess conditions cause a significant drop of the SLA (down to 20% !).</li></ul>There is another way to say it: if your IS is run in such a way that message queues are often full of pending requests, setting up a proper SLA is a very difficult job, because predicting the behaviour (response time) of an overloaded queing system is hard science. It is not enough to add reasonamble margins (such as, promise a 10 minutes response time because the average processing time is 1 minute).There is nothing new here. This experiment confirms what experience or intuition shows. What is interesting (and what surprised me) is the HUGE difference that the computing experiment reveals.I plan to do similar experiments within the (global) entreprise context. I need a model that links the behavior of the IS with that of the company itself. Fortunately, I can rely on the great work (and models) just released by the <a href="http://www.ceisar.org/">CEISAR</a>. The CEISAR is a French initiative, under the patronage of the Ecole Centrale, to create a repository of models and practical knowledge about Enterprise Architecture. A first gem is their global <a href="http://www.ceisar.org/what-we-deliver/publications/download/name/main-concepts.html">model</a> (follow "main concepts" then "Core Business System" on their web site), an attempt to define Enterprise Architecture with 10 key concepts. Another extremely useful piece that is part of the first release is a document about <a href="http://www.ceisar.org/what-is-enterprise-architecture/entity-rules.html">entity modeling</a>. In one of my <a href="http://www.dunod.com/pages/ouvrages/ficheouvrage.asp?id=50092">books</a> I complained that this type of knowledge was not accessible (and could only be obtained from experience). It is nice to see real-life-experts, such as <a href="http://www.club92.net/www.mutlimania.com/letter/index.php?option=com_content&task=view&id=429&Itemid=98">Jean-René Lyon</a>, share their knowledge about such topics.I definitely plan to adhere to CEISAR's terminology and framework for my own future work about IS architecture. One of the most pressing issue (as I have already testified on this blog) is to build a framework/model to explain, discuss, simulate data distribution and synchonization protocol. The only way to make this a relevant topic is to keep a very broad perspective, that includes a model of the coupling between IS and business. The nice conclusion is that this type of work falls neatly between my two topics of interest (cf. my other blog) : IS efficiency and Enterprise efficiency.

Lean Information Systems

Lean Manufacturing is a powerful concept, which is often misunderstood. It was made popular by Toyota’s implementation and Taiichi Ohno’s vi...

9:25 PM

Long term Research Agenda

I have published my <a href="http://claire3.free.fr/yves_research.htm">research agenda </a>on my web site. I will resume my work on OAI this summer to prepare a lecture that I need to give in October. However, my first priority is to translate my book into English. I am currently finishing my computational experiments on Social Networks. Contrary to what I have posted on the first message of this blog, there is a fair amount of common grounds between my two research topics. Mostly, the importance of information propagation ... how is it relevant to business performance, and how does information techhnology help to achieve it. The idea that fast is better, that reactivity is a crucial quality ... is everywhere. However, as the study of LeanSixSigma from an operations research's viewpoint shows, this is not so obvious. Lean-ness, Speed, Reactivity come at a price. It is easy to be convinced that the price is small compared to the benefits, but this argument is rarely heard. This is a thread of thought that I will follow ... It is related both to my old job about Information Systems Efficiency and to my new job of VP in charge of business process optimization and total quality management.

Long term Research Agenda

I have published my research agenda on my web site. I will resume my work on OAI this summer to prepare a lecture that I need to give in Oct...

12:02 AM

Complex Systems and Autonomy

This blog is “moving” very slowly because I am currently much more involved with the theme “<a href="http://organisationarchitecture.blogspot.com/">Information flow and Enterprise architecture</a>”. I am currently working on Social Networks, and it turns out that there are a number of themes which are equally relevant to my two research interests. For instance:<ul><li>I will soon post a review of Robert Axelrod’s book “The Complexity of Cooperation”, which could appear in both my blogs. This book introduces a few key concepts, such as the use of game theory and genetic algorithms to study patterns of (stable) cooperation.</li><li>I attended a few sessions of the conference on “<a href="http://complexsystems.lri.fr/RNSC/tiki-index.php?page=Colloque+CNRS+2007&bl">Complex Systems</a>”, which I thought was appropriate considering that DIS (Distributed Information Systems) are indeed complex systems, and it turns out that social networks, including those that represent the information flows within a company, are the focus of “complex system approaches” as well. The 7th PCRD has made “socially intelligent IT” a hot priority.</li></ul>My program nowadays is mostly to continue my education and read books. I will give a lecture in October for which I would like to extend the OAI research I have already talked about. More precisely, here are the two directions that I will pursue this summer when I resume my experimentation work:<ol><li>Explore different scales and designs of systems to see what kind of influence they have on the behaviour of “smart” routing rules for process control (through message passing). Most of the experiences that I have made so fare try to reproduce business processes from Bouygues Telecom. Since I am using different “Enterprise Simulation Scenarios” in my SIFOA experiments, I plan to reuse this modelling effort. By scale & design, I mostly mean the number of processes, the number of tasks and systems involved in a process, and the interaction topology induced by the SOA architecture.</li><li>Try to model the handling of exceptions, that is, to model the alternate processing path which is used once a component is unavailable to deliver a business process. The difficulty here is that it is often an ad hoc approach (cf. my papers and my books). On the other hand, with the current trend of virtualization, more systematic alternate approaches will become available. I attended an interesting lecture on Autonomic computing from IBM which made the obvious-but-profound point that there is no autonomy without choice, and that choice comes (mostly, in the world of IT) from virtualization. There is little room for improving the handling of exceptional situations and failures with autonomic computing if a traditional architecture (hard links between dedicated resources) is used. On the other hand, in a virtualized world, there are many interesting options (hence a choice) when a server becomes unavailable. </li></ol>This means that I am currently focusing on the points (1) and (2) of my previous list. Following the suggestion from Cedric Nicolas, I just read “the age of spiritual machines” from Ray Kurzweil. It is a fascinating book, with very insightful thoughts, especially about consciousness and intelligence in a machine (or a network of). I am less enthusiastic about his prediction of the future (2009-2019-2099), but it is only a fourth of the book and the first three quarters are enlightening. I will return to the topic of consciousness when I post a review of Kevin Kelly’s book. I also finished a book from Jean-Claude Ameiesen, “La sculture du vivant” (following a suggestion from Pierre Haren) which is totally different (a biology book about programmed self-destruction of cells) but equally fascinating. This type of biology book makes it even more convincing that large scale man-made systems must draw their inspiration from living organisms. There are a number of mechanisms designed to yield both stability and flexibility/adaptability (somehow antagonistic) which are worth reproducing.

Complex Systems and Autonomy

This blog is “moving” very slowly because I am currently much more involved with the theme “ Information flow and Enterprise architecture ”....

4:02 AM

Subscribe to: Posts ( Atom )