Scenario planning in public policy: Understanding use, impacts and the role of institutional context factors Axel Volkery, Teresa Ribeiro European Environment Agency, Strategic Futures Group, Kongens Nytorv 6, DK 1050 Copenhagen K, Denmark a r t i c l e i n f o a b s t r a c t Article history: Received 24 november 2008 Received in revised form 15 july 2009 Accepted 27 july 2009 Scenario planning has formed a growing area of interest on the interface of academia and public and private sector policy-making. While methodological approaches are covered well in the academic literature, less attention has been paid to studying the use, impacts and effectiveness of scenario planning in public policy-making. This article combines preliminary findings from a review of evaluative scenario literature with workshop discussions among scenario practitioners, using environmental relevant policies as a case study. Subject to the nascent evaluative scenario literature our preliminary findings highlight that scenario planning still is executed often in a rather ad hoc and isolated manner and is geared mostly towards indirect decision support such as agenda-setting and issue-framing. The slim evidence base aggravates the assessment, but the potential of scenario planning to prepare public policy-making for the uncertainties and surprises of future developments and better manage complex decisions involving conflicting societal interests is clearly not fully utilized. Political and institutional context factors need to be treated with greater care in the future. Making better decisions under conditions of deep uncertainty does not only require rigorous analysis, but also political will and more stable institutional settings and organisational capacities to build up trust and experience with adaptive, flexible process formats. We synthesize our analysis with a discussion of further research needs. 2009 Elsevier Inc. All rights reserved. Keywords: Scenario planning Decision support Political and institutional context Evaluation 1. Introduction Policy-makers and business leaders often face strategic decisions with uncertain future outcomes. These outcomes often depend on a variety of difficult to predict factors beyond their control. Decision-making under conditions of deep uncertainty is a consequence. Scenario planning has been developed as a method to represent and deal with such deep uncertainties. Over the recent decades, it has formed a growing area of interest on the interface of academia and public and private sector policy-making. A diversity of methodological approaches has been applied in a large number of case studies testing to the vibrancy of the field. Monitoring of the field as, for example, provided by the European foresight monitoring Network1 or the Foresight and Governance Project of the Woodrow wilson Center, 2 reveals a broad diversity of activities. Specific units or departments have been institutionalized in public administrations and international organisations 1. Technological forecasting & Social Change 76 (2009) 1198 1207 This article does not represent any official opinion of the European Environment Agency, but reflects the personal opinion of the authors only. Part of the analysis presented in this paper is based on two internal reports 3. We would like to acknowledge the work of Tom Ling Stijn Hoorens, Michael Hallsworth from RAND Europe and Robert Lempert from RAND in this regard. We would also like to thank the participants of the April 2008 BLOSSOM workshop in Copenhagen and the participants of the 3rd International Seville Conference on Future-oriented technology analysis which took place in October 2008 for useful discussions and comments. Finally we would like to thank the reviewers and editors of this special issue for their helpful comments on earlier versions of this article. Corresponding author. E-mail address: axel. volkery@eea. europa. eu (A. Volkery. 1 See for more information the website: www. efmn. org. 2 See for more information the website: http://wwics. si. edu/subsites/lookingforward/index. htm. 0040-1625/$ see front matter 2009 Elsevier Inc. All rights reserved. doi: 10.1016/j. techfore. 2009.07.009 Contents lists available at Sciencedirect Technological forecasting & Social Change Methodological approaches to scenario planning are covered well in the academic literature. Their use and impacts in the realm of public policy is treated, however, often perfunctorily. Environmental policy-making is a good example. As a field of policy action it is characterized by a high degree of problem complexity and uncertainty about long-term future developments. Problems often do not only unfold over several decades, but also cut across spatial scales which are influenced by a myriad of driving forces. While a lot of scenario work has been conducted so far a systematic analysis of its use, impacts and effectiveness in environmental policy-making is still missing. This article aims to contribute to this discussion through a review of what we call the evaluative scenario literature, introducing a pool of 52 literature pieces that deal with one or several aspects of using scenario planning 2. This article merges the findings from this review with the results from a workshop with environmental scenario practitioners and policy-makers. Our analysis is informed predominantly by a focus on scenario planning in environmental policy and other environmental relevant policy fields. It has been written in the context of the BLOSSOM project of the European Environment Agency. 3 The project aims at building a platform for discussion evaluation and learning amongst environmental scenario practitioners and policy-makers about new forms of long-term strategic policy-making and their enabling conditions. We understand scenario planning broadly as the aim to identify a range of new threats and opportunities that arise across a set of plausible alternative scenarios, describe a range of possible consequences for candidate policies, help discover policy options demonstrably robust to long-term uncertainties and surface some of the blind-spots of an organisations'policy, or strategy 4, 5. Broad participation of societal stakeholders is not an essential requirement, but is becoming more and more a standard practice. 4 At the same time covering a broader range of important social, technological, economic, ecological and political trends has become sort of a mainstream practice, too 6. Our analysis underlines the need to treat political context factors and the institutional embedding of scenario planning with greater care than it has been done so far. Making better decisions under conditions of deep uncertainty requires not only rigorous analysis. Even well-constructed, thoroughly analysed scenarios can be of little use and relevance, if the organisational capacity to absorb them is poor, if there is no political backing or if relevant specifics of the policy-making process have not been taken into account. The distinction between scenarios as products and scenarios as processes is relevant in this context. Process is as important as the product 7. The remainder of this article is organised as follows: Section 2 describes the background and leading questions for the analysis. Section 3 synthesizes the findings from the review of evaluative scenario literature. Section 4 complements this discussion with the findings from workshop discussions and takes the discussion one step further and sketches a first analysis of success factors and barriers to scenario planning in public policy. Section 5 discusses the main findings and Section 6 synthesizes implications for the future development of the evaluative scenario literature. 2. Functions of scenario planning in the policy-making process The scenario planning literature highlights a wide range of decision support functions 8, 9. Ideally, scenario planning helps policy-makers making better sense of changes in their external environment, spotting early warning signals and refining perceptions of existing or emerging problems and corresponding problem-solving strategies 10. Moreover, scenario planning helps surfacing and managing conflicts between diverging societal interests and values and helps finding common ground for future action, which in turn is a key essence of policy-making. Participatory scenario planning also helps mobilizing action by different public and private actors. In the longer run, it can trigger cultural change in the way institutions and organisations approach their long-term future and make them more adaptive to external changes 11, 12. Scenario planning needs to be rooted in a sufficient understanding of the milieu in which political decisions are made. One conceptual framework to structure the oftentimes messy processes of policy-making is the concept of the policy cycle 14. This concept breaks the policy-making process down into several phases: 1. Policy issue identification, i e. to recognize that there is a problem; 2. Policy issue-framing and agenda-setting; i e. to highlight the societal relevance of the problem and underline the need for a response from the political system; 3. Policy measure development, i e. to check for the strengths and weaknesses of different problem-solving strategies, make a final selection and formulate the concrete shape of the measure; 4. Policy measure implementation, i e. to put the measure into practical action; 5. Policy measure effectiveness assessment or policy termination, i e. to identify the effects of the policy measure and evaluate to which degree they deliver according to their objective, and, if necessary, redesign or terminate the measure, which would start a new cycle. 3 The European Environment Agency is a specialised agency of the European union with the prime task of providing targeted, timely, relevant and reliable data and information on the state and prospects of Europe's environment. BLOSSOM is an acronym for Bridging LONG-TERM Scenario and Strategy analysis Organisation and Methods, where organisation refers to the relevance of the organisational and institutional context and methods refer to the need of developing methods to better align long-term scenario and strategy analysis 4 A critical distinction needs to be made between quality and quantity of participation. Depending on the context and purpose of the exercise engaging a handful of people can add more value to the process than engaging several dozens, and vice versa 13.1199 A. Volkery, T. Ribeiro/Technological forecasting & Social Change 76 (2009) 1198 1207 The limitations of the policy cycle concept have been discussed widely. In practice, the process of policy-making does not follow a linear sequencing of stages. Processes run in parallel, overlap, short-cut each other or are left out 15 16. However, the concept provides a useful orientation frame for our analysis. It helps the observation that information needs of policy-making, but also actor constellation and related conflict constellations, differ with regard to the framing, design or implementation of policies. This diversity requires sound governance of scenario planning, as different formats might be necessary to conform to diverging information needs and context conditions. The different functions of scenario planning can be distinguished into indirect and direct forms of scenario-based decision support (17, see Fig. 1). Indirect forms relate to the early phases of policy-making. Here is an opportunity for broader participation of societal stakeholders and open-minded discussions. Increasing the information base supports identifying and framing issues of policy relevance. Moreover, scenario planning can offer a policy risk-free space to visualize, rehearse and test the acceptability of different strategies without being implicated by the actual constraints of day-to-day policy-making. Broader participation improves the relevance and legitimacy of the exercise. The picture changes when it comes to the phase of policy design and implementation. Notwithstanding the fact that many strategic policy documents extend their validity beyond legislative cycles (take the case of national sustainable development strategies for example) a short-term, budget-or policy-cycle related perspective frequently prevails in this phase, alongside serious time and resource constraints and the need to fix working compromises between conflicting societal interests. Policy-makers ask for concrete advice and operational benefits taking into account the limitations to shaping the future 18. These direct forms of scenario planning require delivery of more targeted information and insights on the candidate policies or strategies in questions, eliminating less favorable alternative options and focusing on the preferable ones. Moreover, opportunities for broad-scale participation of societal stakeholders are limited, as the choice among policy-alternatives is politicized a highly process in the end. A couple of questions arise from this discussion: 1. How is used scenario planning: is geared it more towards indirect forms of decision support or more geared towards direct forms of decision support, or is there no clear picture? Which impacts can be observed? 2. Which lessons can be drawn from current practices? How can those in charge of setting up a scenario planning exercise in public policy effectively proceed? Which institutional arrangements might be beneficial to the purposes of scenario planning and which capacities need organisations in the public sector develop? These questions guided our review of the available literature 2 . While numerous alternative methodologies exist for future strategic planning, we limited ourselves mainly to literature on scenario planning, because scenarios provide a potentially very attractive approach to addressing uncertainty and complexity. Furthermore, the focus of our review was on literature that explicitly attempts to evaluate scenario planning approaches, including comparative studies of the strengths and weaknesses of several different exercises and assessments of the impacts scenario planning approaches can have on decision-making processes Fig. 1. Forms of scenario-based decision support. 1200 A. Volkery, T. Ribeiro/Technological forecasting & Social Change 76 (2009) 1198 1207 and organisational performance. To identify relevant literature a number of experts were asked for suggestions. Indexes and the reference lists from these initial items were used to find relevant earlier and subsequent literature. A narrow focus on evaluative scenario literature was kept, but the aim was to be inclusive with regard to what was included under this heading. This process resulted in a selection of fifty-two sources from academic journals, books and book chapters, working papers, policy papers, and research reports. There are sure to be important items missing from this list. Nevertheless, the work included seems to provide a reasonable survey of the main themes in the literature. As part of this analysis we organised a two-day international workshop gathering thirty environmental scenario practitioners from governments international organisations, private sector, public advisory bodies and research organisations as well as a couple of policy-makers to discuss current practice, success factors, obstacles and new ways forward in an open brainstorming setting. Prior to the workshop a questionnaire was distributed among participants to investigate the importance of different objectives when using scenario planning. 3. Synthesizing the available evaluative scenario literature 3. 1. Types of evaluative scenario studies The literature attempting to assess the impacts of scenarios employs different types of evaluative methods. Some studies describe the theoretical benefit of scenarios, some drawing on general psychological understanding of human decision processes and biases some reporting specific laboratory tests of the impact on decisions and some referring to observations of decision processes within organisations 19 21. Some studies provide detailed comparisons of several scenario exercises in order to assess the factors affecting the success of such exercises 17,22 25. A particular type of work is ethnographic studies examining how scenarios are used within organisations 26. In addition, survey instruments in workshop settings are useful tools to measure how different types of scenarios can affect decision makers'understanding of the challenges they face and preferences for response options 27. There are also data available on the types of businesses that use scenarios most often large firms in capital-intensive industries with long (greater than 10 years) planning horizons. However only a handful of studies report empirical results on the central issue how scenario use correlates with an organisation's performance 28. Partly this can be explained with related methodological difficulties since impacts might take a long time to become evident or are difficult to single out from the complexity of influence factors. 3. 2. Impacts of scenario planning The few studies that attempt to evaluate the effectiveness of scenarios find them useful 29,30. We only identified one empirical study that concretely assesses the effects of scenario use on organisational performance: the authors examine two UK industries, water supply and IT consulting, and find that scenario use correlates with increased profit and return on capital 28. A comparison of four scenario exercises related to global change applications suggests climate scenarios are used mostly to support further modeling and analysis, though they can also help frame public debates 17. A limited relevance for decision-making in policy processes is recognized by a review of practice in the broader area of foresight: elaborating upon results of a series of workshops with practitioners and policy-makers the authors conclude that new approaches are necessary to increase the relevance and impact of foresight exercises 31. Several studies report effective outcomes in terms of individual learning. Using scenarios can increase participants'perceptions of their strategic communication and conversation skills. This has been confirmed by studies that gather data on individual participants in a scenario planning project 20. One example is the study of Groves et al. who presented decision aids incorporating scenario concepts in a series of workshops to managers and stakeholders of a California water agency and measured participants'increasing understanding of the challenges posed by climate change and changes in their views on how best to respond to these challenges 27. Views on climate change changed, however, rather slightly, confirming insights from the policy analysis literature that it is very difficult to change core belief systems in the short term. This process requires a longer time perspective. Drawing from the private sector literature, using scenarios to address the challenges facing firms can easily fail for another reason: managers can reject the scenarios because no viable options existed that would enable the business to address the threats the scenarios conveyed 32. Several studies highlight potential shortcomings of scenario exercises or emphasize ways in which they deviate from the practice described in the case study literature. For instance, deductive approaches to scenario development are used commonly. They usually rank driving forces, select the two most important driving forces (i e. those driving forces which are highly uncertain in terms of their future development and could have a decisive impact on the topic at hand) and plot them on two axes to structure the subsequent development of four diverging scenario stories. Much of this literature highlights the need for consensus on the scenario axes in order to foster a common basis of understanding. A close examination of a qualitative scenario project conducted by The netherlands Institute for Spatial Research identified three different and contradictory interpretations of the scenario axes employed 26. The project nonetheless proceeded without consensus by producing multiple publications that treated the axes in different ways. The main impacts of scenarios often result more from the process of developing them rather than from any published product describing the scenarios that were created. This argument is underpinned well by the study of global environmental assessments, which also emphasizes the need for salience, legitimacy, and credibility 22. The value in close collaboration between scenario 1201 A. Volkery, T. Ribeiro/Technological forecasting & Social Change 76 (2009) 1198 1207 developers and users, particularly at the beginning and ending stages of a scenario exercise, can also be shown for scenario exercises related to global change applications 17. Process is an important factor, because it has implications for the extent to which people trust scenarios and thus use them. Trust can based on trust in sources, that is, in the people who develop the scenarios or trust in content, that is, the reliability of the information known to be used in constructing the scenarios. Additionally trust is gained from methodological credibility, i e. confidence in the method used to generate the scenarios, or trust in narrative, i e. the extent to which the scenarios build on existing metaphors and beliefs. Finally, trust in dissemination is important, that is the stature of the people who present the scenarios 11. A number of studies propose ways that government agencies might be organised to make better use of information contained in scenarios. One study for the U s. government suggests several steps for using long-range forecasts more effectively, including: 1) endow a dedicated centre to become a source for high quality, timely reports for Congress; 2) change executive branch and Congressional rules to increase pressure to consider such forecasts; 3) engage public and media with such forecasts more effectively; and 4) establish credible watchdogs to monitor and report how government responds to forecasts 33. In addition, developing networked, small, flexible, task-oriented, managerial teams in government's executive branch overlaid on the existing bureaucracy could significantly enhance the government's ability to analyse and act on scenarios 34.3.3. Treatment of surprise and discontinuities A reason often quoted for using scenario planning as a decision support tool is its ability to reduce overconfidence about the future 35. However, a number of studies argue that it is difficult for scenarios to accommodate or anticipate surprises or discontinuities. The addendum to the Hart Rudman Commission which warned of the dangers of a 9-11-scale terrorist attack, examined 20 scenario studies of U s. national security and found that all tended to focus on extrapolations of current concerns and rarely focused on other possibilities that can produce startling emergent behavior 34. A comparison of the treatment of surprises in 22 scenario studies highlights that seven of the eleven scenarios including discontinuities were exploratory, while four were decision support exercises. All but one exercise with surprises was developed via an intuitive rather than formal process. None of the model-based scenario exercise included surprises 35. In this context another study suggests that standard scenario approaches tend to systematically exclude surprising or paradoxical developments as inconsistent or logical impossible. Building scenarios from trends rather than key driving forces might reduce the combinations that normally become excluded in the scenario axis approach, because of potential inconsistencies. Inductive approaches more easily allow focusing on wild cards, the authors argue, and extend the scenario building process to introduce seemingly paradoxical elements to force a sensitivity analysis on the assumptions underlying claims of inconsistency 36. Inductive approaches, in opposition to deductive approaches, are guided not by a structuring Framework scenarios rather evolve by brainstorming events and spinning larger stories around these events. While this might lead to novel storylines which might challenge existing belief systems of participants more effectively, the process is also less systematic, requires more time and a greater willingness and patience to engage in creative discussions. These conditions are met oftentimes not in the sphere of policy-making, triggering a stronger use of deductive approaches that require less time, seem more logical and easier to communicate. However, one might argue that the higher costs of inductive approaches in the beginning might pay off later in the process given the fact that unmet expectations about the ability of scenarios to deliver novel insights about future developments oftentimes lead to frustration or rejection of the approach by policy-makers. Further analytical efforts are needed but our review did not provide the space. 3. 4. Differences between public and private sector applications Much of the literature on scenario theory and practice focuses on business applications. A number of studies thus examine potential differences between scenario exercises in the public and private sectors. A comparison of five public sector scenario exercises addressing regional development concludes that there is little difference in the actual day-to-day work of developing scenarios between public and private sector applications 23. However, scenario users in the public sector can face more difficult challenges in establishing the client framing the purpose of the engagement, and gaining the participation of the all the relevant parties. Methods that work well in developing scenarios for small groups, well known to the scenario developers, may not work well in developing scenarios that can be used by large organisations or in broad political debates 37. Public sector decision makers may also face particular constraints such as a diversity of legitimate, but competing objectives and societal interests 38. The previously mentioned ethnographic study of scenario axis practice in The netherlands concluded that the diffuse and heterogeneous nature of objectives and interests faced by public agencies may make impossible any consensus about the meaning of scenario axes 26. Difficulties in applying traditional scenario methods among the heterogeneous stakeholders public sector scenario exercises must engage are reported also elsewhere in the literature 17. The traditional scenario axis approach argues against including probabilistic information with scenarios. However, in some public sector applications such probabilities may prove useful. In summarizing the arguments in the climate change community for and against including probabilities with climate scenarios a key study concludes that probabilities may be useful in some situations, in particular when the key variables distinguishing the scenarios are few and quantitative and the potential scenario users are numerous and diverse. Probabilities may be less useful when the scenarios are rich, complex narratives; their purpose is heuristic exploration; and the users are few, similar, and known 39.1202 A. Volkery, T. Ribeiro/Technological forecasting & Social Change 76 (2009) 1198 1207 4. Reviewing practitioners'perspectives 4. 1. Relevance and success of direct and indirect forms of decision support To validate and extend the findings from our literature review, we organised an information exchange with thirty practitioners. On the basis of their personal experience, workshop participants were asked to rate the direct and indirect forms of scenario-based decision support both on a scale from 1 (very low) to 5 (very high) with regard to the specific relevance (importance) and on a scale from 1 (very low) to 5 (very high) with regard to the specific success they have in achieving these objectives. It is not possible to assign statistic significance to the results, since the group of respondents comprised only 16 participants. Due to its representation of different geographical and administrative backgrounds the sample of answers is nonetheless interesting and provides food for further thought. Overall, participants identified a higher relevance of more indirect forms of scenario-based decision support. Stimulating wider debates about possible futures and clarifying an issues importance were ranked as the objectives of highest relevance. Moreover participants also felt that theyweremost successful in achieving these outcomes. Framing a decision-making agendawas scored to be quite relevant butmore difficult to achieve, whereas getting stakeholder engagement and buy in was described as easier to achieve, but of less relevance in the practical day-to-day work. This confirms findings from other studies 31 and our own literature review 2. Scenario exercises are used often as ameans of collecting and synthesizing complex information and creating a common language and shared understanding between different actors 40. Users often value the opportunity to explore contentious issues outside of the usual parameters in a safe space. The process of doing so may persuade key decision makers of the need for further action. It aligns very much with the information requirements of the early phases of the policy cycle. Turning talk into action is however, the critical challenge. Workshop participants were much more cautious in attributing relevance and success here. In particular, scenario planning was considered less relevant and less successful in appraising the robustness of options for future action. A couple of arguments were raised to explain this notion. Scenario planning can easily clash with established routines of political decision-making which are informed by policy path-dependencies. The level of political difficulty is higher for these forms of scenario-based decision support, as they touch upon a number of vested interests around policies, both within government and its core target groups. Those networks of actors are seldom willing to accept changes in core aspects of relevant policies. Therefore it seems consequential to try to restrict the use of an approach that by its very nature aims at challenging these core aspects and provides a platform for voicing conflicting opinions and interests. Furthermore, the overall rationale of scenarioplannnin logic might be at odds with the rationale of policy deliberation: scenario planning follows a rather linear, rational logic of identifying a problem structure, shaping problem solutions and determining the solution that works best across a range of alternative futures. In reality, policy deliberations are shaped oftentimes by power, specific interests and conflict constellations. Resulting decisions might appear irrational when evaluated against the framework of scenarios. Another argument concerned a potential mismatch of scenario formats and information demands by policy-makers. The literature discusses the notion of scenarios often being hollow diamonds that sparkle alluringly but fail to contain real value to the decision-making process 35. Our process came to similar conclusions. In particular this concerns examples of futures thinking that failed to make knowledge useful failed to link it to the real concerns of decision makers, and failed to provide knowledge at the time it was needed. Oftentimes, external experts are commissioned by governments and public administration to produce forward-looking analysis, leading to problems of lack of ownership and trust and a greater motivation to ignore or treat the analysis superficially. Workshop participants also acknowledged the general difficulties of evaluating impacts of scenario planning. However, there was a common agreement among workshop participants that the potential of scenario planning is currently underutilized in dayttoday policy-making. Apparently, a rather ad hoc use is still prevalent. Scenario planning and futures thinking in more general terms, was described to often be isolated a rather activity. 4. 2. Factors determining success Institutional capacity-building was deemed an important requirement for making overall progress. Participants were ask to rate the importance of a number of different success factors (see Fig. 2). Political factors, the level of user-involvement, and the skills and experience of those using the scenario outputs were regarded to be of critical importance. Logically, it would seem that an important aspect of the skills and experiences of those carrying out the scenario exercise is the selection or negotiation of the scenario methodology. It is therefore interesting that the cluster of categories concerning methodologies were rated as lower in importance and that the spread of responses for these categories was relatively high. This perhaps reflects the view expressed in the workshop that a simple tool discussion is not possible as the very notion of an appropriate methodology is contested complex and. Discussions pointed to recent approaches to strengthen capacities for scenario planning and foresight within governments. Examples include: The UK government has institutionalized a Comprehensive foresight Programme which is based in the Government office for Science based in the Department for Innovation, Universities and Skills. Additionally, horizonscanning is anchored in many departments, often in form of dedicated units. 5 Examples of using scenarios to frame and support decision-making include the 5 The Future Analyst Network (FAN-Club) has been established as a permanent network of people dealing with future-related issues in different departments, agencies and the private sector. 1203 A. Volkery, T. Ribeiro/Technological forecasting & Social Change 76 (2009) 1198 1207 UK government's strategy on flood and coastal erosion risk management, which incorporated much of the thinking from the Environment Agency's preceding Flood and Coastal Defence Project. In Sweden, the Swedish Environmental Objectives Council which is responsible for monitoring Sweden's set of long-term environmental quality objectives used a set of four alternative scenarios to generate a range of new, additional policy instruments and measures that could help reaching the objectives. In Finland, a national foresight reporting mechanism has been institutionalized, requiring the Prime ministers Office to release one report per electoral period addressing a range of long-term issues. The Committee for the Future which is appointed by Parliament and is the only parliamentary committee specifically dedicated to general future concerns in a member country of the European union, prepares a statement in response. At the same time, the procedure has included a series of Regional Future Forums. Some critical aspects of institutional arrangements can be distinguished. First, analysis can be conducted by actors within the public administration, or by external parties. Secondly, the organisational mandate of the actors responsible can be either permanent or temporary. Thirdly, a sector-based or a cross-sector approach can be used. Examples can be found for all categories and sometimes practice covers more than one dimension. For example, the UK Foresight programme both develops own analysis and commits external analysis. Its mandate is temporary, and it is based on a cross-sectoral approach. On the other hand, the strategy unit in the Department for Agriculture in The netherlands is an example for an actor in charge of long-term sectoral analysis. A number of other procedural options can be utilized to strengthen the role of scenario planning. Using specific requirements for ex-ante, ex-post or midterm policy evaluation could make long-term scenario planning become a more integrated component of the policy cycle. The European commission's requirements for extended regulatory impact assessment provide for an example. Here different ex-ante evaluations procedures have been integrated into one regulatory policy appraisal approach to improve the quality of regulation, improve stakeholder interaction and ensure a proportionality of costs and benefits. The overall process is coordinated by the Secretariat General of the European commission, and quality assurance should be reached through interadminisstrativ consultation, whereas the assessment as such is carried out by the responsible directorate-generals of the Commission. These assessments are required to assess ex-ante several regulatory options. Demanding a check of important decisions against a set of more alternative, long-term scenarios could help to broaden the focus of assessments, which several evaluation studies found as one weakness of efforts so far 41. Another entry point could be Strategic Environmental Assessment. An example can be found in Scotland, where scenario planning has been part of developing Scotland's 2nd National Planning Framework (NPF), setting out the range of strategic alternatives that were considered as part of the early development of the NPF, and identifying respective environmental effects 42.5. Discussion of findings The evaluative scenario literature remains nascent, making it difficult to come to an elaborated discussion. While there seems to be embedded a lot of tacit knowledge in government and public administration related efforts are reported seldom and analysed. As a consequence, those in charge of scenario planning in public policy, but also those trying to evaluate ongoing practice, can draw on only scant empirical evidence to support their judgement. Findings presented in this article are thus necessarily preliminary. Scenario planning is geared more towards indirect forms of decision support in the early phases of the policy cycle, and a number of successful examples of awareness-raising and issue-framing were raised in discussions. A lot of progress needs to be Fig. 2. The key success factors of scenarios and other futures methodologies. 1204 A. Volkery T. Ribeiro/Technological forecasting & Social Change 76 (2009) 1198 1207 made, however, towards getting scenario planning more fully incorporated into processes of policy design, choice and implementation. While many studies report on individual learning effects, there is a larger gap when it comes to understanding the impacts of scenario planning on overall organisational performance. The literature does not report too many differences between public and private sector applications. But scenario practitioners often confirm that the heterogeneous nature of objectives and interests faced by governments makes it difficult to establish a key client, frame the purpose and gain the participation of all relevant participants. The compartmentalized structure of modern governments is a key barrier to more integrated policy thinking and decision-making which is indeed a key aspiration of scenario planning. In addition, policy-makers and strategists often have not only different time horizons than scenario planners, but also very different attention foci. While this is long known, a mutual confusion about information needs, expectations and capacities to conform to these aspirations continues to exist. Contradictions between product and process functions of scenario planning might aggravate this confusion. Does this assessment mean that future efforts should concentrate rather on indirect forms of scenario-based decision support, since they are easier to implement? Our discussions with scenario practitioners pointed into the different direction. Having an impact on the design and choice of policies remains a litmus test for the relevance of scenario planning. Current conditions in many governments and public administrations constrain the ability to effectively pursue direct forms of scenario-based decision support. Many scenario exercises are commissioned in an ad hoc manner, and usually are executed by external contributors. Follow-up is often lacking, and the role and purpose within the decision-making process is not always clear. It is not remarkable to diagnose a limited impact on political decision-making when the governance conditions for scenario planning are developed not rightfully. The struggle of balancing requests for open-minded broad discussions with requests for clear and targeted input into processes of decision-making is apparent. Scenario planning needs to conform to a variety of information demands, conflict constellations and diverging concerns for scientific credibility, legitimacy and relevance, both within and outside government and public administration. This is a much more complex problem structure than oftentimes assumed. It requires flexible and adaptive scenario formats and processes that vary between open and closed formats of interaction with many or few actors 12. Notwithstanding the context dependency of scenario planning, more and more scholars argue that effective scenario planning therefore needs procedural stability in addition to flexibility. More permanent institutional settings and organisational capacities are required to build up trust and experience create a learning framework to flexibly incorporate new advancements in knowledge, adapt structures and decisions to new insights and trigger more long-term changes in the organisations'overall awareness for constant anticipation, learning and adaptation 12,43. While this need has been recognized in the literature, a more systematic debate around options for the better institutional embedding of those approaches is still largely missing 12. A combination of skills development, capacity-building, undertaking evaluation more regularly, as well as making better use of existing knowledge and knowledge platforms is a useful first step. Creating the right incentives for policy-makers to engage in scenario planning gain trust into the process and build in elements of accountability is a second step which is more difficult to achieve. A number of options come up, when considering the different dimensions of organisational choice. A distinction can be made between hard and soft links. Hard links relate to institutional links and formal, discrete inputs to the policy-making process. Ministries can be obliged by regulation, cabinet decision or voluntary agreement, for example, to undertake scenario planning regularly and report on it, for example through submitting yearly or multi-annual foresight reports. The institutional arrangements in Finland are trend-setting in this regard in particular because Finland involves a modus of interaction between government and parliament. Several studies point to the need of central coordination and a high political backing, ideally at cabinet level or head of government level, which require some form of enactment. This can be combined with developing networked, small, flexible, task-oriented, managerial teams in government's executive branch. Soft links are concerned with creating more informal spaces for discussion and exchange between policymakker and their key stakeholders, in an ongoing mode with few official outputs. Here, the focus is being rather on gradually changing perceptions changing mind-sets and building shared understanding. Different potential structures emerge according to the type of links that are desired. However, there is no uniform approach as diverging political and administrative cultures as well as overall traditions and styles of policy-making and administration will heavily influence the design of institutional arrangements. It is not possible to provide standard answers to questions such as whether it more advisable to go for a centralized approach, with a central unit in charge of coordinating other departmental units, or a rather decentralized approach, with departmental units coordinating themselves. Whether specific foresight units should be located physically within departments for the environment for food, for energy or transport for example, or whether there would be added value in having more flexible, virtual networks that can easily adapt to changing context conditions and information requirements, remains another open question. In the next phase of our project work, we aim at delivering a sample of focused country case studies for selected member countries of the European union to analyse and compare specifically the strengths and weaknesses of different institutional arrangements. 6. Conclusions There is no uniform experience of using scenario planning, and there is no standard approach towards its institutionalization within government and public administration. This is not only natural in a booming field where the practice and literature has been flourishing in recent years, but there will also never be a standard approach. Long-term thinking does not have a uniform relationship with strategy and policy-making. Context is crucial. Many practitioners consider scenario building to be a craft or art of challenging mind-sets in strategic conversations about long-term futures. From this point of view, some general rules of 1205 A. Volkery, T. Ribeiro/Technological forecasting & Social Change 76 (2009) 1198 1207 conduct can be agreed on, but in the end only practice helps to master the field in the end. If on the other hand, scenarios are seen as part of a scientific inquiry, as in the case of global environmental assessments such as the Millennium Ecosystem Assessment or the IPCC, work on the systematization of methods and associated success criteria is needed, and indeed possible. Notwithstanding the context dependency of scenario planning two key conclusions can be drawn from our analysis for further developing the evaluative scenario literature. The first conclusion concerns the need for broadening the empirical base to better understand how and to which effect scenario planning is used and how it can deliver to its promises. The second conclusion concerns the study of institutional arrangements that are put in place to make scenario planning work. In this paper we reported findings from a scoping phase of a project that in its following phases will hopefully contribute to filling this gap. There are important claims in the literature about scenario's ability to affect the cognition of individuals and groups by reducing overconfidence, improving willingness to consider a range of plausible futures, increasingly willingness to think innovatively about robust strategies and to more effectively detect early warning signals. But only a handful of studies have evaluated the actual practice. Numerous case studies suggest that scenarios can improve the performance of organisations, but few studies have tested these claims. Oftentimes the importance of the context and process of scenario creation is emphasized. But there is little evidence to connect these insights to resulting performance of the participating organisations. The evaluative scenario literature should improve alongside these lines. Many public sector applications may require a more systematic connection between scenarios and recommended decisions than do private sector applications. Forthcoming efforts should include conducting empirical studies of the impacts of scenarios on organisational performance and their key drivers; and develop a deeper understanding of how the context and process of scenario creation affects their impacts. If and how probabilistic information should be used with scenario planning in public policy would be another concern. While probabilities may prove a hindrance in many business sector applications, others argue that the large, diverse audiences for public policy scenarios often require some type of information about relative risks. In addition, the question of a potential correlation between scenario approaches and impacts and effectiveness of use require further evaluative efforts. The postulate of some studies that inductive approaches seem to perform relatively less well in comparison to more inductive approaches when it comes to anticipating discontinuity and surprise is interesting. However inductive approaches are considered also often to be too time-consuming and ambiguous in their outcomes, highlighting the need for further methodological improvements. While the ability of a scenario planning exercise to deal with discontinuities and surprise factors is influenced by methodological design aspects, it may be more dependent on the skills and abilities of those carrying in out and the degree to which they are empowered formally to take participants through novel and provoking discussions. Getting policy-makers to trust the credibility, relevance and legitimacy of the approach will require more than methodological refinements. More stable institutional settings are needed to experiment with flexible process formats that can adapt analytical efforts to specific information needs and conflict constellations. Whether the intention is to challenge mind-sets of political decision makers or to drive specific decisions (and either might be the case) scenario practitioners need to pay more attention to windows of opportunity, the areas of maximum uncertainty and doubt, and the issues most amenable to influence. Institutional capacities are treated often superficially in the discussion. Further efforts should be directed to learn more systematically from cases of good practice and to synthesize this knowledge base within a comparative analysis. Sound process management needs to exploit existing routes to influence, such as policy networks, the media, business schools or schools of government. There are sufficient examples to suggest that scenario planning is an appropriate tool in the right circumstances but that one needs to better account for practical limitations. In the short run, it can be helpful where the choice of futures tools is appropriate, where the scenario builders and developers are sufficiently skilful and representative, and where the intended target audience of the work is engaged, understood and sufficiently responsive. Given the manifold limitations within the process of policy-making, however, modesty is required about the claims made on behalf of scenario planning. References 1 Wavfrify, The role of futures thinking in government: report of the February FAN Club meeting, Prepared for the Horizon scanning Centre, 2008, available at: http://www. foresight. gov. uk/Horizon%20%Scanning%20centre/Fanclubnews/Feb2008. asp (accessed 7th july 2008. 2 European Environment Agency, Looking Back on Looking Forward. A Review Of Evaluative Scenario literature, Technical Report 3/2009, Copenhagen, EEA, 2009.3 R. J. Lempert, S. Hoorens, M. Hallsworth, T. Ling, Looking Back on Looking Forward: A Review Of Evaluative Scenario literature, RAND WR-564-EEA, 2008.4 D. Mercer, Robust strategies in a day, Manage. Decis. 35 (1997) 219 223.5 P. Wack, Scenarios: shooting the rapids, Harvard Bus. Rev. 63 (6 nov.//Dec.,1985) 139 150.6 T. Henrichs, M. Zurek, B. Eickhout, K. Kok, C. Raudsepp-Hearne, T. Ribeiro, D. van Vuuren , A. Volkery (in prep. Scenario development and Analysis for Forward-looking Ecosystem Assessments. In: MA Millennium Ecosystem Assessment (in prep. Ecosystems and Human Well-being: A Manual for Assessment Practitioners, forthcoming. 7 B o'Neil, S. Pulver, S. Vandeveer, Y. Garb, Where next with global environmental scenarios? Environ. Res. Lett. 3 (2008) 045012.8 P. Bishop, A. Hines, T. Collins, The current states of scenario development: an overview of techniques, Foresight 9 (1)( 2007) 5 25.9 R. Bradfield, G. Wright, G. Burt, G. Cairns, K. Van der Heijden, The origins and evolution of scenario techniques in long range business planning, Futures 37 (8)( 2005) 795 812.10 R. J. Lempert, S w. Popper, S. C . Bankes, Shaping The next One hundred Years: New methods for Quantitative, Longer-Term Policy analysis, 2003, RAND MR-1626-RPC. 11 C. Selin, Trust and illusive force of scenarios, Futures 38 (1 )( 2006) 1 14.12 E. A. Eriksson, M. Weber, Adaptive foresight: navigating the complex landscape of policy strategies, Technol. Forecast. Soc. Change 75 (4)( 2008) 462 482.13 A. Volkery, T. Ribeiro, T. Henrichs, Y. Hoogeveen, Your vision or my model? Lessons from developing participatory land use scenarios on a European scale, Syst. Pract. Action Res. 21 (6)( 2008) 459 477.14 M. Howlett, M. Ramesh, Studying public policy, Policy Cycles and Policy Subsystems, Oxford university Press, Oxford , 2005.15 T. R. Dye, Understanding Public policy, Prentice hall, Englewood Cliffs, NJ, 1984.1206 A. Volkery, T. Ribeiro/Technological forecasting & Social Change 76 (2009) 1198 1207 16 P. Sabatier, H. J. Jenkins-Smith, The advocacy-coalition-framework. an assessment, in: P. Sabatier (Ed.),Theories of the Policy Process, Westview Press, Boulder, CO.,1999, pp. 117 166.17 E. A. Parson, V. R. Burkett , K. Fisher-Vanden, D. W. Keith, L. O. Mearns, H. M. Pitcher, C. E. Rosenzweig, M d. Webster, Global-change scenarios: their development and use, Sub-report 2. 1b of Synthesis and Assessment Product 2. 1 by the U s. Climate change Science Program and the Subcommittee on Global Change Research, Department of energy , Office of Biological & Environmental Research, Washington, 2007.18 J. Rosenhead, Robustness analysis: keeping your options open, in: J. Rosenhead (Ed.),Rational analysis for a Problematic World: Problem Structuring Methods for Complexity, Uncertainty, and Conflict, Wiley, Chichester, UK, 1989.19 P. J. H. Schoemaker, Multiple scenario development: its conceptual and behavioral foundation, Strateg. Manage. J. 14 (1993) 193 213.20 T. J. Chermack, et al. Exploring the relationship between scenario planning and perceptions of learning organization characteristics, Futures 38 (7)( 2006) 767 777.21 Y. Garb, S. Pulver, S. Vandeveer, Scenarios in society , society in scenarios: towards a social scientific analysis of storyline-driven environmental modeling, Environ. Res. Lett. 3 (2008) 045015.22 W c. Clark, R. B. Mitchell, D. W. Cash, Evaluating the influence of global environmental assessments, in: R. B. Mitchell, W c. Clark, D. W. Cash, N m. Dickson (Eds. Global Environmental Assessments: Information and Influence, MIT Press, Cambridge, 2006.23 J. Oglivy, E. Smith, Mapping public and private scenario planning: lessons from regional projects, Development 47 (4)( 2004) 62 72.24 M. B. A. van Asselt, Perspectives on Uncertainty and Risk: The Prima Approach to Decision support, Springer, Berlin, 2000.25 United states Commission on National Security in the 21st century, New world Coming: American Security in the 21st century, Study Addendum, 1999.26 S. A. Van't Klooster, M. B. A. van Asselt, Practicing the scenario axis technique, Futures 38 (1)( 2006) 15 30.27 D. Groves, D. Knopman, R. Lempert, S. Berry, L. Wainfan, Presenting Uncertainty About Climate change to Water Resource Managers, 2008, RAND Corporation, TR-505-NSF. 28 R c. C. Phelps, S. C. Kapsalis, Does scenario planning affect performance? Two exploratory studies, J. Bus. Res. 51 (2001) 223 232.29 J. C. Glenn, T. J. Gordon, Factors Required for Successful Implementation of Futures research in Decision making, Millennium Project publication, 2001, download at: www. millennium-project. org/millennium/applic-exsum. html (accessed 10/02/09). 30 P. C. Light, The Four Pillars Of high Performance: How Robust Organisations Achieve Extraordinary Results, Mcgraw-hill, New york, 2005.31 O. da Costa, et al. The Impact of foresight on Policy-making: Insights from the FORLEARN Mutual learning Process, Institute for Prospective Technological Studies, 2006, Joint research Centre/European commission. 32 G. P. Hodgkinson, G. Wright, Confronting strategic inertia in a top management team: learning from failure, Organ. Stud. 23 (2002) 949 977.33 B. White, Will policy makers use long range forecasts? Paper prepared at the invitation of the Woodrow wilson International Center for Scholars Foresight and Governance Project, 2002.34 L. Fuerth, Strategic Myopia, the Case for Forward Engagement, The National Interest, Spring , 2006, pp. 57 62.35 P. Van Notten, A m. Sleegers, M. B. A. van Asselt, The future shocks: on discontinuity and scenario development, Futures 35 (5)( 2003) 423 443.36 Theo J. B. M. Postma, Franz Liebl, How to improve scenario analysis as a strategic management tool? Technol. Forecast. Soc. Change 72 (2005) 161 173.37 R. Lempert, Can scenarios help policymakers be both bold and careful? in: F. Fukuyama (Ed.),Blindside: How to Anticipate Forcing Events and Wild Cards in Global Politics, Brookings Institution Press, WASHINGTON DC, 2007.38 T. Ling, Decision making in the public sector, in: G. Ringland (Ed.),Scenarios In public Policy, John Wiley, Chichester, 2002.39 D. G. Groves, R. J. Lempert, A new analytic method for finding policy-relevant scenarios, Glob. Environ. Change 17 (2007) 73 85.40 J. A. Dewar, Assumption-based Planning: A Tool for Reducing Avoidable Surprises, Cambridge university Press, Cambridge, 2002, and New york. 41 J. Hertin, K. Jacob, A. Volkery, Policy appraisal, in: A. Jordan, A. Lenschow (Eds. Innovation in Environmental policy? Integrating the Environment for Sustainability, Edgar Elgar Publishing, London, 2008, pp. 114 133.42 Scottish government, National Planning Framework 2 SEA Annex to the Environmental Report: Assessment of Strategic Alternatives, Edinburgh, 2008.43 K. Van der Heijden, Can internally generated futures accelerate organisational learning? Futures 36 (2)( 2004) 145 159. Axel Volkery, Phd, is a project manager for policy and scenario analysis at the European Environment Agency in Copenhagen, Denmark. His research interests are integrated environmental assessments, long-term policy analysis and institutional arrangements for effective scenario planning. He has authored or co-authored several contributions to academic journals and books on these topics. Teresa Ribeiro heads the Strategic Futures Group of the European Environment Agency in Copenhagen, Denmark. She has authored co several scenario and foresight publications at the European and international level. Teresa has worked in environmental policy issues for over 20 years in various countries and for European union Institutions. She is a member of the Global Agenda Council on Strategic foresight of the World Economic Forum. 1207 A. Volkery, T. Ribeiro/Technological forecasting & Social Change 76 (2009) 1198 1207

< Back - Next >

Overtext Web Module V3.0 Alpha
Copyright Semantic-Knowledge, 1994-2011