Exploratory Modeling and Analysis, an approach for model-based foresight under deep uncertainty Jan H. Kwakkel, Erik Pruyt Faculty of technology, Policy, and Management, Delft University of Technology, 2600 GA, Delft, The netherlands a r t i c l e i n f o a b s t r a c t Article history: Received 14 may 2011 Received in revised form 12 july 2012 Accepted 27 august 2012 Available online 29 october 2012 Exploratory Modeling and Analysis (EMA) is an approach that uses computational experiments to analyze complex and uncertain issues. It has been developed mainly for model-based decision support. This paper investigates the extent to which EMA is a promising approach for future oriented technology analysis (FTA). We report on three applications of EMA, using different modeling approaches, in three different technical domains. In the first case, EMA is combined with System Dynamics (SD) to study plausible dynamics for mineral and metal scarcity. The main purpose of this combination of EMA and SD is to gain insight into what kinds of surprising dynamics can occur given a variety of uncertainties and a basic understanding of the system. In the second case, EMA is combined with a hybrid model for airport performance calculations to develop an adaptive strategic plan. This case shows how one can iteratively improve a strategic plan through the identification of plausible external conditions that would cause the plan to performpoorly. In the final case EMA is combinedwith an agent-based model to study transition dynamics in the electricity sector and identify crucial factors that positively and negatively affect a transition towards more sustainable functioning of the electricity sector. This paper concludes that EMA is useful for generating foresights and studying systemic and structural transformations despite the presence of a plethora of uncertainties, and for designing robust policies and plans, which are key activities of FTA. 2012 Elsevier Inc. All rights reserved. Keywords: Future-oriented technology analysis Exploratory Modeling and Analysis Deep uncertainty System dynamics Adaptive policymaking Agent-based modeling 1. Introduction Future-oriented technology analysis (FTA) is understood as an umbrella label for various approaches to explore future developments, including technology forecasting, technology intelligence, future studies, foresight, and technology assessment 1. In their own ways each of these approaches is used for analyzing technological developments and their potential consequences. Technology refers both to physical artifacts as well as to social practices that specify how these artifacts can be used. Thus, technological systems can be decomposed in the physical components as well as the social components including institutions. The various fields covered by the umbrella term FTA have at their disposal a wide variety of methods, techniques, and approaches. A subset of these approaches relies, at least in part, on mathematical and computer models. The reason for using models might be understood in light of the rise of Newtonian mechanics and its success in predicting a wide array of different phenomena 2. It brought together the celestial realm and the sublunar realm in a single explanatory framework 2, 3. Moreover, it showed that theory was‘a far more effective means than observation for precisely characterizing complex orbital motions physical theory gained primacy over observation for purposes of answering specific questions about the world'3. Over the course of the eighteenth century, Newtonian mechanics was interpreted by Laplace as a clockwork universe after the success of the theory of gravity in accounting for complex deviations from Keplerian motion became fully evident 2, 3. If the mechanisms of the clock are known, any future state of the clock can be predicted. Similarly, if the Technological forecasting & Social Change 80 (2013) 419 431 Corresponding author. Tel.:++31 15 27 88487. E-mail addresses: j. h. kwakkel@tudelft. nl (J. H. Kwakkel), e. pruyt@tudelft. nl (E. Pruyt. 0040-1625/$ see front matter 2012 Elsevier Inc. All rights reserved. http://dx. doi. org/10.1016/j. techfore. 2012.10.005 Contents lists available at Sciverse Sciencedirect Technological forecasting & Social Change mechanisms underlying a phenomenon are known perfectly, one could predict the future development of this phenomenon. With the rise of computers and user friendly software, more and more mechanisms can be, and are codified into computer models. However, the use of models to make predictions can be seriously misleading if there are profound uncertainties. The solar system of planets is a relatively small system the sun and the eight planets and can be observed very well, and thus its behavior can be predicted with great accuracy. However, for many other phenomena, such as the world's climate, or systems in which humans are involved, the situation is different. In these cases, there are many components and mechanisms that interact in a variety of ways, and the system can only partly be observed. The use of predictive models for such systems is problematic. There have been scientists who have realized this. Some claim the forecast is always wrong 4, others say all models are wrong 5, and yet again others qualify arithmetic for such systems as useless 6. Such comments raise the question whether models can be used at all in decision-making under uncertainty. In their agenda setting paper on FTA Porter et al. 1 note that there are many irreducible uncertainties inherent in the forces driving toward an unknown future beyond the short termand predictions need not be assumed to constitute necessary precursors to effective action. In other literature, this is called deep uncertainty 7, 8, or severe uncertainty 9. It can be understood as a situation where one can incompletely enumerate multiple possibilities without being able or willing to rank order the possibilities in terms of how likely or plausible they are judged to be 8. There is a need for model-based support for the design of robust strategies across this spectrum of irreducible uncertainties. The RAND Corporation developed a technique called Exploratory Modeling and Analysis (EMA) tailored to this. EMA aims at offering decision support even in the face of many irreducible uncertainties, by systematically exploring the consequences of a plethora of uncertainties ranging fromparametric uncertainties (e g. parameters ranges), over structural uncertainties (e g. different structures and models), to method uncertainties (e g. different modeling methods) using computational models as scenario generators. This paper explores the potential of EMA for FTA. It thus explicitly addresses one of the FTA challenges identified by Porter et al. 1 by assessing how EMA could contribute to adaptive foresight 10 under deep uncertainty. Particular attention is given to the potential of EMA in offering decision support for shaping systemic and structural transformation. The paper is structured as follows. Section 2 provides more background on EMA. To further elucidate what EMA is and in order to assess the potential of EMA for FTA, three case studies are reported in Section 3. Section 4 is a discussion of the results of these cases and their implications for FTA. Section 5 contains the conclusions. 2. Exploratory modeling and analysis Various scientific fields including the environmental sciences, transportation research, economics, and the political sciences, are involved in providing model-based decision support. In these various fields, people are grappling with the treatment of deep and irreducible uncertainty while using models. A common theme across these fields appears to be a shift away from predictive model use towards more explorative model use 6 11,12. Exploratory Modeling and Analysis (EMA) is a research methodology that uses computational experiments to analyze complex and uncertain systems 12,13. Porter et al. 1, in their agenda setting paper on FTA, explicitly mention EMA as being of potential interest to FTA. To our knowledge, the potential of EMA for FTA has however not been investigated yet. This paper can be seen as an attempt to do so. EMA can be useful when relevant information exists that can be exploited by building models, but where this information does not allow specifying a single model that accurately describes system behavior. In this circumstance models can be constructed that are consistent with the available information, but such models are not unique. The available information is consistent with a potentially infinite set of plausible models, whose implications for potential decisions may be quite diverse. A single model run drawn from this set provides a computational experiment that reveals how the world would behave if the various guesses this single model makes about the various irreducible uncertainties are correct. By conducting many such computational experiments, one can explore the implications of the various guesses. EMA is the explicit representation of the set of plausible models the process of exploiting the information contained in such a set through a large number of computational experiments, and the analysis of the results of these experiments 12,13. EMA is focused not narrowly on optimizing a (complex system to accomplish a particular goal or answer a specific question, but can be used to address‘beyond what if'questions, such as Under what circumstances would this policy do well? Under what circumstances would it fail? and what is the range of plausible future dynamic developments of a phenomenon of interest? Under what circumstances can we expect which dynamic developments? Because of this focus, EMA stimulates‘out of the box'thinking and can support the development of adaptive plans or policies. EMA is first and foremost an alternative way of using the available models, knowledge, data, and information. In making policy or planning decisions about complex and uncertain problems, EMA can provide new knowledge, even where strict model validation is impossible. For example, EMA can be used for existence proofs or hypothesis generation, by identifying models that generate atypical or counterintuitive behavior. Knowing that a system can exhibit such behavior can change the debate or open up new directions for the design of targeted solutions. Another example is the case where there is ample data available but also disagreement or uncertainty about which data to use. EMA can be used to identify the extent to which the choice of data influences the model outcomes. Instead of debating the choice of the right data, the debate can then shift to the development of policies or plans that produce satisfying results across the alternative sets of data. Other possible uses of EMA include the identification of extreme cases, both positive and negative, in order to get insight into the bandwidth of expected outcomes, and the identification of conditions under which significant shifts in performance can be expected. All these examples rely on the fact 420 J. H. Kwakkel E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 that policy or planning debates can often be served even by the discovery of thresholds, boundaries, or envelopes that decompose the entire space of uncertainties into subspaces with different properties. That is, partial information can inform policymaking or planning even when prediction and optimization are not possible by using the available partial information in a systematic and transparent way. Many well-established techniques, such as Monte carlo sampling, factorial methods, and optimization techniques, can be usefully and successfully employed in the context of EMA 7, 13 15. In this paper, we argue that by using models differently, the challenges associated with decision-making under deep uncertainty can largely be overcome. Instead of trying to predict, the models are used to explore what could happen and what policies would hold across various uncertainties. In this way, decision-making can proceed despite the presence of deep uncertainty, for decisions can be designed to be robust across the explored range of possible futures. By supporting the systematic exploration of the complete space of combinations of uncertainties, EMA addresses one of the often mentioned shortcomings of foresight, namely its impressionistic character 10.3. Illustrations of EMA In this paper, EMA is illustrated via three cases. These cases differ in application domain, the type of models used, and the purpose of the study. In this way, together these cases offer a good overview of what EMA is about, and what can be done with it. Each of the cases is related to important societal challenges. The first case explores uncertainties related to the availability of minerals/metals that are crucial for the sustainable development of all societies. The second case shows how EMA can be used to develop adaptive plans for guiding airport development. Airports are a major driver for regional and national economic development. Future uncertainty is increasing because contextual conditions are less stable, new technical solutions are emerging, and evaluation criteria are contested (e g. noise and emissions versus economic benefits) 16. EMA offers a suitable technique to explore the potential implications of these uncertainties and assists in developing a plan that can adapt over time to how uncertainties unfold. The third case presents an EMA study into transition pathways for the Dutch electricity system. Recent contextual developments constitute a backdrop of change for the Dutch electricity system. Institutional change driven by liberalization, changing economic competitiveness of the dominant fuels, new technologies, and changing end-user preferences regarding electricity supply are some examples of these developments. EMA is used to explore plausible transition trajectories in the face of these developments given technological uncertainty about investment and operating costs, and fuel efficiency of various alternative technologies; political uncertainty about future CO2 abatement policies such as emission trading; and socioeconomic uncertainty about fuel prices, investment decisions of suppliers, and load curves. 3. 1. Mineral scarcity The first case explores uncertainties related to the availability of minerals/metals that are crucial for the sustainable development of all developed and developing societies. Potential mineral/metal scarcity poses a serious challenge for civil protection in at least three ways 17,18: 1. Many crucial minerals and even some base metals such as copper 19 22 and lead 23 may in a few decades become more difficult and expensive to mine and process, possibly posing threats to satisfy increasing overall demands for those metals in order to sustain a higher standard of living of an increasing world population. 2. The disparity between the expected exponential growth of metal demand and the expected limited growth of metal supply (especially of crucial low-volume metals such as rare earth metals that are required in ever bigger quantities for many innovative technologies and electronics) may result in temporary and/or chronic scarcity. 3. Strategic or speculative behavior of countries that have a quasi-monopoly on the extraction of (rare earth) metals may seriously hinder the transition of modern societies towards more sustainable ones. The asynchronous dynamics of supply and demand, aggravated by reinforcing behaviors and knock-on effects, is a breeding ground for acute and/or chronic crises 18: how and when these dynamics and crises may materialize remains highly uncertain. EMA is used in this case to create insight into plausible dynamics. That is, it serves as an existence proof generator. 3. 1. 1. Model Traditionally, System Dynamics is used for modeling and simulating dynamically complex issues and analyzing their resulting nonlinear behaviors over time in order to develop and test structural policies 24, 25. Under conditions of deep uncertainty, long time horizons, and high dynamic complexity, a more exploratory use of models is called for 26. Mineral and metal scarcity is characterized by long time horizons diverging beliefs and ideas about system functioning, and complex interactions between supply, demand, substitution, and recycling, necessitating a more exploratory approach. Causal loop diagrams are used often to communicate feedback loop structures included in System Dynamics models. Fig. 1 displays the main feedback loops of a generic simulation model developed to create insight into the types of dynamics that can occur. The two recycling loops displayed in red are reinforcing loops. The linked extraction sector feedback loops are controlling loops. This suggests that if recycling takes off, it is limited intrinsically by the extraction sector loops. These extraction sector loops in turn are limited by the steeply increasing costs of extraction. More details on the model can be found in 18. The model has been implemented as a System Dynamics model using the Vensim software 27.421 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 This small and simplistic System Dynamics model was developed in about one day in close collaboration with a mineral/metal expert , based on his mental model of the underlying structure of the mineral/metal system 18. The objective of the joint modeling endeavor was twofold:(i) to explore plausible dynamics of mineral/metal abundance/scarcity, and (ii) to identify, describe and visualize interesting scarcity scenarios for the client. Both objectives were achieved at first by means of traditional System Dynamics modeling and manual exploration of the influence of key assumptions, changing one assumption at a time. At a later stage, the model was used as a scenario generator for EMA, allowing the automatic and simultaneous exploration of many uncertainties and assumptions. This EMA use of the model is reported below. 3. 1. 2. Uncertainties The future evolution of the extraction and recycling is intrinsically uncertain. The presented model allows exploring alternative evolutions across the various uncertainties. Table 1 gives a high level overview of the key uncertainties that are taken into consideration. Note that we explore across both parametric variations and what typically would be considered more Table 1 High level uncertainties. Name Description Ranges Parametric uncertainties A wide variety of parametric uncertainties are explored, including the lifetime of mines and recycling facilities, the initial values, and behavioral parameters such as price elasticity and desired profit margins. Typically plus and minus 50%of the default value Orders of time delays There are various time delays, such as building of new recycling capacity and mines. First order, third order and tenth order, 1000th Nonlinear lookups There are various nonlinear relations modeled with lookups. Examples include learning effects, the impact of shortage on price, and substitutions in case of shortages. These nonlinear relations are varied by changing the start, end and slope. Start, end, slope Fig. 1. Causal loop diagram of the scarcity model 18.422 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013 ) 419 431 structural variations. The exploration is handled using the python programming language 28, utilizing the Vensim DLL 27,29 to parameterize the model, run the model and extract the results. 3. 1. 3. Analysis of results Fig. 2 shows the dynamics for 5 different outcomes of interest. It illustrates clearly the wide variety of dynamics that can occur. It also shows that under specific conditions cyclical behavior can emerge, with overshoots in supply, followed by undershoots, in turn causing a highly dynamic market price. The behavior of this one outcome is investigated in some more detail in order to investigate the emergence of crises and price cycles. Fig. 3 shows the behavior of the relative price for a thousand runs. This figure shows more examples of cyclical behavior and it appears that the cycles can become worse over time. This display however also shows the need for further analysis: the individual runs are difficult to trace in this plot. Therefore, there is a need for data reduction techniques. One way of analyzing the results is to identify runs that share the same dynamic behavior over time. The behavior over time can be understood as being a concatenation of atomic behavior patterns 30. By combining atomic behavior patterns with the sign of the first order derivate, six different behaviors are possible (i e. positive logarithmic, negative logarithmic, positive exponential, negative exponential, positive linear, negative linear. Each time series can then be converted into a concatenation of these six patterns. Clustering then takes place on the basis of the concatenations. If a hard clustering algorithm is used, which is to say that the entire concatenation needs to be identical, then Table 2 is the result. This table shows the wide variety of behaviors that the model can generate by sampling across the various uncertainties. A more detailed analysis was performed in order to assess whether the cyclical behavior arises out of a particular combination of uncertainties. First, in order to identify crises behavior we look at the first order derivative of the results. Using this, the results are classified as showing crises behavior if the first order derivate is anywhere higher then 1, and not showing crises behavior Fig. 2. Results of 100 runs. The second column of the figures shows a Gaussian kernel density estimate of the end states of the runs. It provides insight into the relative density of the results across the range of values. Note that one should be careful not to interpret these densities in a probabilistic sense, since deep uncertainty does not warrant probabilistic interpretations. 423 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 otherwise. Next, we tried to identify subspaces in the overall uncertainty space that show a high concentration of crises runs using the Patient Rule Induction Method 31 33. We were unable to find any subspace, indicating that crises behavior does not originate in a particular subspace but arises out of particular unique combinations of uncertainties. That is, crisis behavior is generated in this model by‘perfect storm'type combinations of parameters. This finding is troublesome to decision-making, for it implies that crises may be difficult to predict based on the monitoring of various exogenous developments. More thorough analysis of the results is needed still for even roughly 6000 behaviors in case of 50,000 runs are still unwieldy for supporting decision-making. There is an emerging field that studies the clustering of time series data. A wide variety of methods and techniques are being explored 34. Application domains include chemical process monitoring and control, and gene expression research. This area of research might contain useful techniques for further reducing the results and supporting the interpretation. Along another dimension, more research is currently ongoing for particular metals, such as copper, indium, and tantalum. The purpose of these studies, which are actively using EMA, is to create insight into the possible dynamics and to explore possible strategies to cope with or prevent certain undesirable dynamics. 3. 2. Adaptive planning for airport development The air transport industry operates in a rapidly changing context. Changes in ownership structure, initiatives like Single European Sky and the Open Skies treaty between the United states and Europe, the introduction of new aircraft such as the Airbus A380 and the Boeing 787, and advances in Air Traffic Management (ATM) technology radically alter the functioning of the sector 35. Airports are a crucial element in this system and are major drivers of regional and national economies. Their long-term planning is therefore of crucial importance 36. Amsterdam Airport Schiphol has been working over the last couple of years on a plan for guiding its long-term development 37,38. In this section, a stylized version of this decision-making problem is explored. The purpose of EMA in this case is to help in the development of an adaptive plan for the long-term development of Amsterdam Airport Schiphol that is robust across the wide variety of uncertainties experienced by the airport. 3. 2. 1. Model The uncertainty airport planners face is located mainly in the external environment affecting the airport. The uncertainty about the internal workings of an airport is comparatively small. Therefore a single fast and simple model, utilizing existing tools for aspects of airport performance calculations has been developed 15. Table 3 gives an overview of the components that make up this fast and simple model. This model is a general purpose model, by parameterizing it for the specifics of a particular airport (runway locations, etc. the performance of that airport can be calculated. Table 2 Identification of behaviors. Number of runs Number of behavior patters 1000 371 5000 1214 10,000 2042 20,000 3386 30,000 4547 40,000 5511 50,000 6404 Fig . 3. Evolution of market price for a 1000 runs. 424 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 3. 2 . 2. Uncertainties A wide variety of uncertainties are important in long-term airport planning. Table 4 gives an overview of the major uncertainties that are explored in this case. For more details on the parametric ranges of the various uncertainties see 15. By picking a functional form for each uncertainty a scenario generator is specified. In total, 48 different generators are possible. Each generator in turn has its own parametric ranges over which one can sample. 3. 2. 3. Analysis of results One key challenge for airport planners is to design a plan for guiding the future developments of the airport that is robust with respect to the future 36. For Amsterdam Airport Schiphol the design of such a robust plan is particularly challenging because its serves as a secondary hub of KLM-Air france, implying that this airline can move operations to its primary hub, Charles de Gaul, easily. Moreover, Schiphol is located in a wind prone area, necessitating a runway layout that covers the various wind directions. At the moment, Schiphol is considering expanding the airport by adding a new runway that is to become operational in 2020. Moreover, a participatory process has resulted in the agreement that no more than 510,000 operations can be scheduled at Schiphol in 2020. Up to 70,000 short haul operation are to be relocated from 2015 onwards to the existing airport Eindhoven, and Lelystad Airport which is to be developed in the coming years. Using a conjugant gradient optimization algorithm the lower bound and the upper bound for each of the performance indicators can be calculated. These bounds are calculated across the 48 scenario generators and their associated parameter ranges. The column‘static plan'in Table 5 shows the results of this analysis. Looking at the various outcome indicators, and in particular the ratio of capacity to demand, it is clear that the outlined plan does succeed not in robustly guiding the future development of the airport. There is the potential for significant under use of the provided infrastructure, or a significant overshoot in the negative external effects (noise, external safety and emissions. A sensible redesign of the plan would be to make it dynamically adaptive 35,36, 44. The construction of a new runway and the moving of operations are in this approach not planned for a particular moment in time, but are triggered by the evolution of external conditions. Thus, a new runway is built only if there is sufficient demand, or problems with wind conditions. However, preparatory actions, such as land use reservations, designs for the runway, etc. are taken, minimizing the time required to realize the change. To address the potential overshoot of negative external affects this modified dynamic adaptive plan is complemented with a stricter slot allocation mechanism. The column‘adaptive plan'in Table 5 shows the performance bounds for this redesigned plan. It is clear that this plan has a much narrower bandwidth in expected outcomes across all the uncertainties. It is thus better able to guide the future developments of the airport in light of the uncertainties. To investigate in more detail the difference in performance between both plans, and to create insight into the specific range of conditions under which one plan would perform better than the other, Fig. 4 has been generated. The performance difference is calculated using the Euclidian norm and the normalized performance vectors for each plan 15. This figure shows that under the conditions most favorable for the static plan, if demand is high and/or the increase of aircraft size is high, there is a big advance in using the dynamic adaptive plan. Conversely, if the growth of demand is minor, the static plan performs slightly better. This figure Table 3 Tools integrated in the fast and simple model for airport performance analysis. Airport performance aspect Tool Capacity FAA Airfield Capacity Model (FCM ) an extension of the classic Blumstein model 36 39. Noise Area Equivalent Method (AEM) a model that approximates Integrated Noise Model results 40. Emissions Emission Dispersion Modeling System (EDMS) the FAA required tool for emission analysis 41. Third party risk Methodology developed by the National Air Traffic Services (NATS) for third-party risk 42,43 the NATS methodology has been extended to apply to multiple runways 49,50. Table 4 Major uncertainties adapted from 15. Name Description Range Demand Change in demand, the curves can be parameterized in various ways Exponential growth, logistic growth, or logistic growth followed by logistic decline Wide body vs. narrow body aircraft mix Change in aircraft mix, the curves can be parameterized in various ways Linear or logistic change Population Change in population density, the curves can be parameterized in various ways Logistic growth or logistic growth to a maximum followed by logistic decline ATM technology Change in air traffic management technology, the curves can be parameterized in various ways Exponential or logistic performance increase Engine technology (noise/emissions) Change in air traffic management technology, the curves can be parameterized in various ways Exponential or logistic performance increase Weather Percentage of change in days with severe wind conditions per year. This affects the availability of runway configurations. -1%+4%425 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 could serve as a starting point for slightly modifying the outlined dynamic adaptive plan, for example by modifying the exact values that are used to trigger actions, or by modifying the stricter slot allocation regime. 3. 3. Identification of plausible transition pathways for the future Dutch electricity generation system Recent contextual developments constitute a backdrop of change for the Dutch electricity system. Institutional change driven by liberalization, changing economic competitiveness of the dominant fuels, new technologies, and changing end-user preferences regarding electricity supply are some examples of these developments. In this case we use EMA to explore plausible transition trajectories in the face of these developments given technological uncertainty about investment and operating costs, and fuel efficiency of various alternative technologies; political uncertainty about future CO2 abetment policies such as emission trading; and socioeconomic uncertainty about fuel prices, investment decisions of suppliers, and load curves. Various alternative developments for these uncertainties are specified. The consequences of each of these alternative developments are assessed using an agent-based model 45 of the Dutch electricity system. The outputs are analyzed using CART 46, a classification tree algorithm, in order to reveal arch-typical transition trajectories and their conditions for occurring. 3. 3. 1. Model Electtrans is based an agent simulation model, which explicitly focuses on multiple actor groups within the electricity system, most importantly the end users and the generation companies. Four groups of end-users are represented in the model which are industrial users, commercial users, horti-/agricultural users, and households. It is possible to name two major supply options for all actor groups, i e. using electricity supplied via central generation, and adoption of distributed generation options for self-generation. There are two grid-based options in the model: gray electricity and green electricity. Various distributed generation options are also available, such as wind turbines and gas engine CHPS. Generation companies are mainly responsible for short-term operation, as well as long-term management of their generator park. The short-term operation involves unit commitment decisions, and price bidding in the electricity market, which are directly related to load dispatching to take place in the market. The long-term decisions are related to capacity investment and decommissioning, which decisions are expected mainly based on lifetime of the technology used in a generation unit. A unit at the end of it is lifetime and/or an old unit making loss Table 5 Performance bounds of the static and the adaptive plan. Outcome indicators Static plan Adaptive plan Size of noise contour after 30 years (km2) 13.2 63.8 10.2 47.4 Cumulative Average Casualty Expectancy (ACE ) 0. 9 2. 7 1. 1 2. 3 Ratio practical capacity to demand after 30 years 0. 25 2. 48 0. 89 1 . 1 Maximum ratio practical capacity to demand 0. 9 2. 48 0. 52 1. 1 Accumulated latent demand (flights) 0 5, 058,504 0 8, 290,622 Cumulative CO emission (kg) 21 520.9 195,729. 1 19,773. 9 103,899. 5 Fig. 4. Performance difference of the Adaptive Plan compared to the Static Plan for that combination of uncertain parameters that most favor the Master plan. Below 0 the static plan is better, above 0 the Adaptive Plan performs better. 426 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 can be decommissioned. Generation companies'expansion decisions are driven mainly by profit expectations, and are dependent on forecasts about fuel prices, demand, active generation capacity connected to the grid, and feasible investment options. 3. 3. 2. Uncertainties Table 6 presents an overview of the uncertainties that are explored in the EMA study. In total 13 uncertainties are explored across the specified range. Most of uncertainties are multiplier factors that will be used to alter the base value of the corresponding parameters. For example, assume the investment cost of wind turbine is 100, and the Investment Cost Factor for wind is 0. 8, then the model will be initialized with an investment cost of 100×0.8=80. To be more precise, it is not the initial investment cost we are altering, but the expected future investment cost towards which the option evolves during the time horizon of the simulation. 3. 3. 3. Analysis of results Fig. 5 shows a performance envelope for five outcome indicators. Total generation: total amount of electric energy generated. Total fossil: total amount of electric energy generated using fossil fuels as the energy source. Total non-fossil: total amount of electric energy generated using non-fossil fuels as the energy source (renewables and nuclear. Capacity: total installed power generation capacity. Price: average price of electricity on the central grid. The figure shows the upper and lower bounds that are encountered across the 15,000 runs. The figure also shows the distribution of outcomes at the end of the runtime. It appears from these results, that there is a limited development of non-fossil generation, suggesting that under most uncertainties a transition towards more sustainable generation does not take place. To provide insight into how the various uncertainties jointly determine outcomes, a classification tree was made based on the results. Classification trees are employed a frequently data mining technique 46. They are used to predict class membership based on a set of attributes. In the context of this paper we used the uncertainties (Table 6) as attributes. As class we used the terminal value for the fraction of fossil fuel-based generation. This terminal value was split. If it was lower or equal to 0. 6, it is coded as 0, else it is coded as 1. Fig. 6 shows a classification tree that results from this analysis. The tree was generated using the open source data mining package Orange 47. This is A c++library with python bindings to many useful data mining and machine learning algorithms. This tree can be used to see how the uncertainties jointly affect the extent of a transition towards more sustainable generation we have to identify the leaves coded‘0, 'and then follow the path from that leaf back to the root to identify which uncertainties jointly produce the cases belong to that particular leaf. This reveals that the cases where the fraction of fossil fuel-based generation is lower than or equal to 0. 6, typically occur when the price of fossil fuel is high, while the costs for investing in non-fossil fuels are low. For example, the extent to which a transition occurs is depended critically on the gas price. If the gas price is very low the only real hope for a more sustainable generation is a high carbon price (left most branch in Fig. 6). Looking at the tree, we also conclude that under most developments, the future generation of energy will not be very sustainable. That is, in most cases, the fraction of fossil based generation in the final year is higher than 0. 6. Thus, it appears that if the Dutch government aims at a transition towards more sustainable generation, the current policies are not effective enough across a large part of the uncertainties. 4. Results and implications for FTA This paper started from the observation that model-based decision support under conditions of deep uncertainty is problematic. We noted that Porter et al. 1 shortly touched upon this issue and on the potential of EMA for FTA. More specifically, the authors ask whether EMA can be used to facilitate the development of robust strategies even in the presence of many Table 6 The major uncertainties and their ranges. Name Description Range Investment cost factor Multiplier factor to alter the future investment cost of new generation options 0. 6 1. 25 Operational cost factor Multiplier factor to alter the future variable operating costs of a technology 0. 6 1. 25 Coal and gas price increase percentage Yearly fractional increase in coal prices 0. 002 0. 03 Demand growth fraction Yearly fractional change in the demand of end users 0 0. 03 Load slope change fraction Yearly fractional change in the slope of the load duration curve-0. 01 0. 01 Planning horizon of the generation companies Upper bound for the planning horizon of the generation companies. Planning horizon for each generation company is initialized randomly using a uniform distribution with a lower (i e. 5 years) and an upper bound) 6 12 Mean return on investment of generation companies Average expected return -on-investment for the generation companies 0. 1 0. 25 427 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 irreducible uncertainties inherent in the forces driving toward an unknown future beyond the short term and the large number of scenarios encompassing the spectrum of those uncertainties. However, no careful assessment of EMA for FTA has taken place yet. We illustrated EMA for FTA using three cases. Below we shortly discuss each case, before drawing more generic implications of these cases on the potential of EMA for FTA. These cases differed in the modeling paradigm that was used, in the application domain, and in the type of problem being investigated. The first case showed how EMA can be combined with System Dynamics to investigate the types of behavior that can occur with respect to mineral and metal shortages. In this case both parametric and Fig. 5. Performance envelopes and distribution of end states for five outcome indicators. Fig. 6. Classification tree of fraction fossil-based generation. 428 J. H. Kwakkel E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 some structural uncertainties were taken into account. The case also showed that further research on time-series clustering is necessary to facilitate the interpretation of results. Finally, it showed that if a high tech company is dependent on specific minerals and/or metals, the results of the case could be used to identify early indicators of, for example, cyclic pricing behavior. The second case used an ensemble of hybrid models to facilitate the design of a good plan for shaping and guiding the future development of an airport. The case illustrated how through the use of nonlinear optimization techniques a performance bandwidth could be established across all the uncertainties. This bandwidth showed unacceptable behavior of the basic plan, necessitating the redesign of the plan. A basic redesign, drawing on adaptivity and flexibility already showed a significant shrinkage of the performance bandwidth. To investigate the conditions under which one plan would outperform the other, a further analysis was presented. These results could be used for further improving the adaptive plan. The third case illustrated how EMA can be combined with agent-based models. In the case, we investigated the transition patterns that could occur and which combinations of uncertainties resulted in which transition pattern. The case thus showed how complex and highly uncertain phenomena such as transitions, can still be treated despite the presence of uncertainties. Moreover, it illustrates how one can identify the ranges of uncertainties that are favorable to a desirable transition and which ranges are not. In particular the use of the classification tree in order to create insight into how uncertainties are mapped to classes of outcomes is useful. These three cases show that EMA can be of use to FTA. FTA aims at offering systemic considerations on future developments for dynamically complex issues. The comprehensive exploration of the consequences of combinations of uncertainties that can be offered by EMA is an important component of such future-oriented systemic thinking. All three cases illustrate this systemic exploration, while in particular the first and third case demonstrate how this can be combined with nonlinear dynamic models (System Dynamics and Agent Based Modeling respectively), which are more appropriate for the types of systems and phenomena FTA applies to. FTA intends to guide policy and decision-making by helping in anticipating and shaping future developments. The second case demonstrates how EMA can be used for guiding decision-making on plans that shape the long-term development of an airport. That is, EMA can help organizations in preparing for and guiding their adjustment, adaptability and ability to shape responses to challenges and transformations. A central issue in many FTA projects is how to cope with a multiplicity of worldviews and values, diverging or even conflicting understanding of how a system is working, and different sources and types of information and data. EMA offers practitioners a model-based method for handling such situations. Rather than developing a single or a small number of model-based estimates for a phenomenon of interest, EMA allows practitioners to develop an inclusive ensemble of models that captures the breath and richness of the multiplicity of worldviews, the different ways of understanding a system, and utilizes the plethora of information sources available. Moreover, EMA can also be used for creatively imagining possible futures and subsequently selecting interesting ones, by combining it with scenario discovery 48. In light of all this, EMA thus appears to be a useful addition to the portfolio of methods and techniques available to FTA practitioners. 5. Conclusions The aim of this paper was to investigate the potential of EMA for FTA. Theoretically, the potential of EMA to FTA is its ability to cope with a multiplicity of deep and irreducible uncertainties in the analysis of decision-making problems and in the process of developing robust strategies for addressing these problems. The cases presented and discussed have shown that EMA can be used to handle diverse types of uncertainties in combination with three quite distinct modeling approaches. Moreover, the cases have shown how EMA can be applied to different domains. All three cases, but in particular the second case, showed how the ability to cope with uncertainties can help in iteratively developing dynamic adaptive strategies that are robust across a large part of the uncertainty space. Uncertainty is recognized increasingly as being a major problem for the use of models in decision-making. The prime example being the role of uncertainty in relation to models used in the context of climate change debates. EMA can have profound implications for the way in which uncertainty is treated and models are being used to support decision-making. Where traditionally, often the uncertainties in the inputs to models are reduced as much as possible, in order to come to a best estimate of model outcomes, EMA shows how one can embrace the full range of uncertainties on the input side to models. EMA can also be used in case there is uncertainty about models, while focusing on the consequences decision-makers care about most: the model outcomes. EMA can for example be used to iteratively reduce the expected bandwidth of model outcomes as in the second case presented. Alternatively, EMA offers the potential to support the process of creatively imagining possible futures, a purpose for which it was used in the first case study. The techniques used in each of the tree cases do not exclude each other. Classification trees can for example be used to define more precisely trigger values for a dynamically adaptive plan or the tree can be used to understand which type of behavior pattern emerges under which combination of uncertainties. The three cases also illustrate the need for combining EMA with machine learning or data mining techniques for otherwise, the problem of incompletely taking into account uncertainty is being replaced by an information overload problem. That is, the systematic exploration of a wide variety of uncertainties produces large datasets that need to be analyzed further using machine learning or data mining techniques in order to extract decision relevant information from it. Porter et al. 1 argued that foresight exercises could not comprehensively explore the full range of scenarios that is encompassed by the many irreducible uncertainties encountered when developing effective action, and wondered whether EMA could alleviate this constraint. In the three cases presented in this paper we have shown that EMA can be used for that purpose. Future research avenues include elaborating on the use of EMA for designing dynamic adaptive policies and the use of EMA for 429 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 scenario discovery. Another major avenue of research is on the communication of EMA that results to policy-makers and FTA practitioners. EMA addresses the problem of deep uncertainty by systematically exploring over the uncertainties, potentially resulting in an information overload. The effective analysis, visualization, and communication of EMA insights are thus of crucial importance for its successful real world application. References 1 A l. Porter A w. B.,G. Clar, J. F. Coates, K. Cuhls, S w. Cunningham, K. Ducatel, P. Van der Duin, L. Georgehiou, T. Gordon , H. Linstone, V. A w. J. Marchau, G. Massari, I. Miles, M. Mogee, A. Salo, F. Scapolo, R. E. H. M. Smits, W. Thissen, Technology futures analysis: towards integration of the field and new methods, Technol. Forecast. Soc. Chang. 71 (2004) 287 303.2 D. Jamieson, Prediction in society, in: D. Sarewitz, R. j. Pielke, R. Byerly (Eds. Prediction: Science, Decision making, and the Future of Nature, Island Press, WASHINGTON DC, 2000.3 G. Smith, Newton's philosophiae naturalis principia mathematica, in: E. N. Zalta (Ed.),The Stanford Encyclopedia of Philosophy, 2008.4 W. Ascher, Forecasting: An Appraisal for Policy makers and Planners, Johns hopkins university Press, Baltimore, 1978.5 J. D. Sterman, All models are wrong: reflections on becoming a systems scientist, Syst. Dyn. Rev. 18 (2002) 501 531.6 O. H. Pilkey, L. Pilkey-Jarvis, Useless Arithmetic: Why Environmental Scientists Can't Predict the Future, Columbia University Press, New york, USA, 2007.7 R. J. Lempert, S. Popper, S. Bankes, Shaping The next One hundred Years: New methods for Quantitative, Long-term Policy analysis, RAND, Santa monica, California, 2003.8 J. H. Kwakkel, W. E. Walker, V. A w. J. Marchau, Classifying and communicating uncertainties in model-based policy analysis, Int. J. Technol. Policy Manag. 10 (2010) 299 315.9 Y. Ben-Haim, Information-Gap Decision theory: Decisions under Severe Uncertainty, 2nd ed. Wiley, New york, 2006.10 E. A. Erikson, K. M. Weber, Adaptive foresight: navigating the complex landscape of policy strategies, Technol. Forecast. Soc. Chang. 75 (2008) 462 482.11 D. Sarewitz, R. A. Pielke, R. Byerly, in: Prediction, Island Press, Washington, D c, 2000.12 S. Bankes, Exploratory modeling for policy analysis, Oper. Res. 4 (1993) 435 449.13 D. B. Agusdinata, Exploratory Modeling and Analysis: A Promising Method to Deal with Deep uncertainty, in: Faculty of technology, Policy, and Management, Delft University of Technology, Delft, 2008.14 J. H. Miller, Active nonlinear tests (ANTS) of complex simulation models, Manag. Sci. 44 (1998) 820 830.15 J. H. Kwakkel, The Treatment of Uncertainty in Airport Strategic planning, Faculty of technology, Policy and Management, Delft University of Technology, Delft, 2010.16 E. Störmer, B. Truffer, D. Dominguez, W. Gujer, A. Herlyn, H. Hiessl, H. Kastenholz, A. Klinke, J. Markard, M. Mauer, A. Ruef, The exploratory analysis of trade-offs in strategic planning: lessons from regional infrastructure foresight, Technol. Forecast. Soc. Chang. 76 (2009) 1150 1162.17 J. Kooroshy, C. Meindersma, R. Podkolinski, M. Rademaker, T. Sweijs, A. Diederen, M. Beerthuizen, S. de Goede, in: Scarcity of Minerals, A Strategic Security Issue, The hague Centre for Strategic Studies, The Hauge, 2010.18 E. Pruyt, Scarcity of minerals and metals: a generic exploratory system dynamics model, in: T. H. Moon (Ed.),The 28th International Conference of the System Dynamics Society, 2010, Seoul, Korea. 19 E. Alonso, J. Gregory, F. Field , R. Kirchain, Material availability and the supply chain: risks, effects, and responses, Environ. Sci. Technol. 41 (2007) 6649 6656.20 A. Valero, A. Valero, Physical geonomics: combining the exergy and Hubbert peak analysis for predicting mineral resources depletion, Resour. Conserv. Recycl. 54 (2010) 1074 1083.21 W. Auping, The uncertain future of copper, in: An Exploratory System Dynamics Model and Analysis of the Global Copper System in The next 40 Years, Delft University of Technology, Delft, 2011.22 J. H. Kwakkel, W . Auping, E. Pruyt, The uncertain future of copper: dynamic scenario discovery under deep uncertainty, Technological forecasting and Social Change,(under review. 23 R. U. Ayres, On the practical limits to substitution, Ecol. Econ. 61 (2007) 115 128.24 J. Forrester, Principles of Systems, Wright-Allen Press, Cambridge, MA, 1968.25 J. D. Sterman, Business Dynamics: Systems thinking and Modeling for a Complex World, Mcgraw-hill, Boston, MA, 2000.26 E. Pruyt, C. Hamarat, The influenza A (H1n1) v pandemic: an exploratory system dynamics approach, in: M. H. Lee (Ed.),The 28th International Conference of The System Dynamics Society, 2010, Seoul, Korea. 27 Ventana Systems Inc, in: Vensim Reference manual, 2011.28 G. van Rossum, in: Python Reference manual, CWI, 1995.29 Ventana Systems Inc, in: Vensim DSS Reference Supplement, Ventana Systems, Inc, 2010.30 D. N. Ford, A behavioral approach to feedback loop dominance analysis, Syst. Dyn. Rev. 15 (1999) 3 36.31 R. J. Lempert, D. G. Groves, S. Popper, S. Bankes, A general analytic method for generating robust strategies and narrative scenarios, Manag. Sci. 52 (2006) 514 528.32 D. G. Groves, R. J. Lempert, A new analytic method for finding policy-relevant scenarios, Glob. Environ. Chang. 17 (2007) 73 85.33 J. H. Friedman, N. I. Fisher, Bump hunting in high-dimensional data, Stat. Comput. 9 (1999) 123 143.34 T. W. Liao Clustering of time series data a survey, Pattern Recog. 38 (2005) 1857 1874.35 J. H. Kwakkel, W. E. Walker, V. A w. J. Marchau , Adaptive airport strategic planning, Eur. J. Transp. Infrastruct. Res. 10 (2010) 227 250.36 R. de Neufville, A. Odoni, Airport Systems: Planning, Design, and Management, Mcgraw-hill, New york, 2003.37 Schiphol Group, in: Lange Termijn Visie op de Ontwikkeling van de Mainport Schiphol: Een Wereldwijd Netwerk Voor een Concurrerende Randstad Long-term vision on the Development of the Mainport Schiphol, 2007.38 Luchtvaartnota Rijksoverheid, in: Concurrerende en duurzame luchtvaart voor een sterke economie Air Transport White paper Competitive and Sustainable Air Transport for a Strong Economy, 2009.39 FAA, Upgraded FAA Airfield Capacity Model, Federal Aviation Administration, Washington D c, 1981.40 FAA, FAA Aerospace Forecast Fiscal years 2008 2025, U s. Department of transportation, Federal Aviation Administration, 2008.41 FAA, in: Emissions and Dispersion Modeling System User's Manual, Federal Aviation Administration, Office of Environment and Energy, WASHINGTON DC, 2009.42 P. G. Cowell, R. Gerrard, D . S. Paterson, A Crash Location Model for Use in the Vicinity of Airports, National Air Traffic Services Ltd. London, UK, 1997.43 P. G. Cowell, P. B. Foot, R. J. Gerrard, D. Kent, S m. Manson, A. Rivoire, in: A Methodology for Calculation Individual Risk due to Aircraft Accidents Near Airports, National Air Traffic Services Ltd, London, UK, 2000.44 C. Hamarat, J . H. Kwakkel, E. Pruyt, Adaptive Robust Design under Deep uncertainty,(under review. 45 G. Yücel, Analyzing Transition Dynamics, Delft University of Technology, Delft, 2010.46 L. Breiman, J. H. Friedman, C. J. Stone, R. A . Olshen, Classification and Regression Trees, Wadsworth, Monterey, CA, 1984.47 T. Curk, J. Dem ar, Q. Xu, G. Leban, U. Petrovic, I. Bratko, G. Shaulsky, B. Zupan, Microarray data mining with visual programming, Bioinformatics 21 (2005) 369 398.48 B. P. Bryant, R. J. Lempert , Thinking inside the box: a participatory computer assisted approach to scenario discovery, Technol. Forecast. Soc. Chang. 77 (2010) 34 49.49 S. J. Heblij, Development of a runway allocation optimisation model. Delft, Delft University of Technology. M. Sc. Thesis, 2004.50 S. J. Heblij, R. A a. Wijnen, Development of a runway allocation optimisation model for airport strategic planning, Transportation Planning and Technology 31 (2 )( 2008) 201 214.430 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431 Jan Kwakkel is a postdoctoral researcher at Delft University of Technology. He received his MSC. and Ph d. degrees from Delft University of Technology. In addition he studied philosophy at Leiden University. His research focuses on the treatment of uncertainty in model-based decision support. He has worked on cases in various domains including air transport, port planning, fresh water supply in The netherlands, world water scarcity issues, and energy transitions. Next to this research, he also has an interest in scientometrics and techmining. Erik Pruyt is Assistant professor of System Dynamics and Policy analysis at the Faculty of technology, Policy and Management of Delft University of Technology. He obtained his master's degree in Commercial Engineering and Phd degree from the Faculty of economics, Social and Political sciences & Solvay Business school of the Free University of Brussels. His research focuses mainly on the multidimensional dynamics of complex uncertain systems, from short-term crises to long-term transitions. 431 J. H. Kwakkel, E. Pruyt/Technological forecasting & Social Change 80 (2013) 419 431

< Back - Next >

Overtext Web Module V3.0 Alpha
Copyright Semantic-Knowledge, 1994-2011