Forecasting and Assessment Methodst that was held in Seville on 13 14 may 2004, organised by the Institute for Prospective Technological Studies (IPTS) of European commission's Directorate General Joint research Centre.
In addition, the idea was to analyse possible overlapping fields of practice among technology foresight, forecasting, intelligence, roadmapping, and assessment.
the participants were asked for an assessment of the process and whether it had been useful for them to participate.
The concluding section provides a critical assessment of the potential of Adaptive foresight which leads us to highlighting some of the methodological challenges that need to be tackled in the future.
and their contributions to be positioned in a comprehensive framework. 7 A fact-based foundation is thus as crucial for the credibility of foresight as a critical assessment of the sources of knowledge.
14 where the complex forecasting and assessment type of approach initially 12 Dixit and Pindyck 23 and Copeland and Antikarov 24 have prepared two very useful texts on real options theory. 13 See for instance the emphasis put on learning processes in protected spaces which is core to the approaches of Strategic
Niche Management 25 and Transition Management 26.14 FANTASIE Forecasting and Assessment of New Transport Technologies and Systems and their Impact on the Environment,
assessment and methodological challenges In a nutshell, we see Adaptive foresight as a promising attempt to circumvent the aforementioned Collingridge Dilemma.
Change 69 (2002) 929 951.31 K. H. Dreborg, E. A. Eriksson, Best practice guide for assessment methods, Report D21 of the DG VII
Methodologically, the processwas based on the Internet-based solicitation and assessment of research issues, the deployment ofrobust Portfoliomodeling (RPM) in the identification of promising research issues,
2. Assessment of research issues Researchers December‘05 Mid-january‘06 3. Assessment of research issues Industrial leaders Three last weeks of January‘06 4. Initial
In total, well over 200 Researchers from the participating countries submitted research issues. 3. 2. 2. 2. Assessment of research issues from the research perspective.
Within each of the 23 sub-areas, some 10 to 50 Researchers provided assessments while more than 200 Researchers in total took part in the assessment activity.
The questionnaire was open from December 2005 until Mid-january 2006. For each issue, Researchers were asked first to assess the issue with regard to Novelty (i e.,
moreover, to describe how they would like to contribute to a possible project later on. 3. 2. 2. 3. Assessment of research issues from the industrial perspective.
and a total of some 50 Industrial leaders participated the assessment phase. The Industrial leaders assessed the issues with regard to Industrial relevance
the Project Team analyzed issues based on the assessment data. For each issue, key statistics were calculated (e g.,
rather, it was employed for the purpose of synthesizing the wealth of information that was collected from the experts during the assessment phase,
and for drawing attention to those issues that were deemed particularly interesting in view of this assessment.
consequently, the RPM analysis was carried out by identifying alternative portfolios of 7 research issues that could be regarded as attractive in view of incomplete preference information about the relative importance of the assessment criteria.
so that they could become familiar with the proposed research issues, experts assessments and corresponding CI values. A separate set of background materials was prepared for each workshop
Then, the participants were invited to comment on the proposed solicited issues and their assessments (approx. 1. 5 h),
Thus, the funding organizations could see how the results based on the representatives of their own country may have differed from those of all expert assessments.
A particularly valuable aspect of bottom up consultation processes is that the solicitation and assessment of research issues,
More evidence-based approaches to the assessment of regulatory quality allow a review of the effectiveness of policy tools used in practice
Here, legal frameworks as an object of assessment are part among the set of other policy instruments, like R&d funding,
and are put in the context of possible evaluation and assessment technologies. Table 1 Evaluation matrix:
This procedure allows a rough assessment of the possible regulatory challenges caused by the dynamics in science
and requires further methodological efforts. 3. 1. 3. General assessment Science and especially technology indicators are a possible source to detect challenges for the regulatory framework in the future.
General assessment of the scope and limits of methodology: The scope of science-and technology-based indicator approaches is certainly in detecting possible fields
whose data permits the assessment of the future needs for and impacts of regulations and standards.
-related aspects, the acceleration and reduction of costs of certification and conformity assessments and generally the decrease in the amount of governmental regulations.
Based on these assessments, it was also possible to identify whether a sector suffers under the burden of too many standards
The comparison between actual standardisation activities and the general assessment of the future relevance of the different types of standards exposes some discrepancies,
especially regarding quality and compatibility standards. 3. 2. 4. General assessment In general, there is only limited experience in the use of surveys for identifying future needs for standards and regulation.
Hence this assessment mainly depends on the experiences collected during the recent past and some previous studies cited above:
& Social Change 75 (2008) 496 516 General assessment of the scope and limits of methodology In contrast to other strategic aspects of organisations, assessing the future needs for standards
Table 2 presents the assessment of the 21 telecommunication-related technologies expected to diffuse mostly within the next ten years.
the assessment of R&d support and standardisation as a more diffusion oriented policy instrument correlate very high,
whereas the statistical connection between R&d support and regulation is rather vague. 3. 3. 4. General assessment In general,
but also for the ex ante assessment of the impacts of regulations standards. Furthermore, the small-scale Delphi survey focusing on the future demand for standards in the ICT area confirmed the general applicability of this approach,
General assessment of the scope and limits of methodology: Delphi surveys are flexible and can be applied in all areas, covering all possible future driving forces,
and confronting them with the assessment of the average opinion. The problem is how to guarantee for the detection of strategic answers,
and assessment of methodologies Although we cannot refer to a broad sample of regulatory foresights, we have been able to collect relevant experiences from some selected studies
sometimes also including an assessment of their possible impacts. Table 3 constitutes an attempt to summarise the information included in the methodologies section and the results and experiences collected.
However, this is only a starting point of an assessment of regulatory foresight methodologies. Despite the availability of a number of foresight methods, there is a scope
Otherwise, this approach produces rather biased assessments. The same is true for Delphi surveys. Consequently, the combination of indicator-based approaches,
and assessment of regulatory foresight methodologies Methodology Type Data requirements Strengths Limitations Indicators Quantitative also providing qualitative information Adequate science
and the organisation Detect insights of specific needs for future regulation High cost and time-consuming Qualitative Assessment of future relevance of regulation,
This short methodological assessment of the few different regulatory foresight methodologies discussed already makes obvious that a simple transfer from applying the methodologies to identify emerging science
10.1016/j. techfore. 2008.02.002 1. Lacunae and prospects of assessment and alignment tools for emerging science and technology For innovation to succeed actor alignment in the form of innovation chains from laboratory to products
and assessments and forecasts have become more reliable. However, in an age of strategic science and high-investment projects decision makers need to identify possible and promising directions and options and influence technology emergence in advance.
These are challenges for current strategic technology intelligence and forward-looking assessment tools. This is especially the case for the recent European Networks of Excellence
In contrast, analyses of assessment practices of researchers and start-ups (who constitute the larger part of Frontiers) seemrare.
assessment tools (market forecasts; knowledge of the technology and market drivers) are generally uncertain 9, 21,22.
Scientists undertake assessments all the time; these assessments are functioning if not always characterised by breadth of focus (a broader view of the field) and depth of vision (i e. possible applications in the long term.
There is general resistance against linearity towards applications imposed upon research: linearity contradicts the open-endedness
Literature in the management of innovation, expectations management and sociology-of-technology fields has stressed repeatedly that for assessments during early stages of technological emergence, more‘open-ended'
Beyond a diagnosis of the situation and suggestions, few assessment tools seem to have been developed and made available to actors.
/7 Cf Rip et al 2005 30‘Assessment'and‘alignment'can be used somewhat interchangeably where they refer to tools that help assessing actions on the way to an anticipated future-tools for‘anticipatory coordination'(learning curves of‘disruptive technologies';‘
allowing the assessment of whether the current strategy is optimum or a transition is needed to another strategy open to this particular actor.
but also as part of the ongoing assessments which need to be evaluated based on dynamics of path emergence.
Section 4 described and explored a use of MPM as support for the articulation and ongoing assessment of Strategic research Areas based on dynamics of the field as a whole.
Tools for assessment/alignment have been discussed in bodies of literatures as diverse and heterogeneous as: strategic management of S&t;
and can be integrated into FTA ACTIVITIES enhancing the quality of assessment/alignment activities. We mapped initial, potential multiplicity paradigms with path characteristics,
It can be linked with assessments of hype cycles, by contrasting the hype surrounding particular paths with the kinds of path-typical challenges that can be anticipated.
New technology Foresight, Forecasting & Assessment Methods, Seville, May 2004, pp. 13 14.28 T. Fleischer, M. Decker,
New technology Foresight, Forecasting & Assessment Methods, Seville, May 2004, pp. 13 14.29 S. Kuhlmann, et al.
Karlsruhe, June 1999.30 A. Rip, T. Propp, R. Williams, G. Spinardi, P. Laredo, A. Delemarle, NEST-SSA 508929‘Assessment Tools for New and Emerging science
Vision Assessment of Molecular Machines, J. Scientometrics 70 (3)( 2007. 42 U. Meyer, C. Schubert, Die Konstitution technologischer Pfade.
Netherlands on assessment of new and Emerging s&t, specializing in biomedical applications of micro and nanofluidics.
Impacts assessment; Logic model; Networking; Actors'alignment 1. Introduction The present article presents results to date2 from research leading towards the production of a Phd thesis entitled Assessing the contribution of Foresight to a more participatory knowledge society.
and assessment approach for foresight exercises has been developed to date. This is a consequence of the fact that most foresight activities,
Thus evaluations typically focus on assessments of whether or not these goals have been attained. The evolution of a common evaluation and assessment framework for foresight exercises
however, could facilitate the identification of good practices irrespective of their specific objectives and levels of implementattion aswell as facilitating the benchmarking and coordination of policies for socioeconomic development.
However, a common evaluation and assessment approach is not possible if evaluations are based on the specific objectives
Table 1 groups together many of the reported objectives and impacts associated with foresight exercises under the headings of‘intermediate'and‘lower level'goals. 5. Approach for developing an impact assessment framework It is possible
New technology Foresight, Forecasting & Assessment Methods, Seville, May 13 14 2004.10 A. Havas, Terminology and Methodology for Benchmarking Foresight programmes,
New technology Foresight, Forecasting & Assessment Methods, Seville, May 13 14 2004.16 Technology Futures analysis Methods Working group, Technology Futures analysis:
New technology Foresight, Forecasting & Assessment Methods, Seville, May 13 14 2004.17 M. Ladikas, M. Decker, Assessing the Impact of future-Oriented Technology assessment, Paper 1 in proceedings EU
New technology Foresight, Forecasting & Assessment Methods, Seville, May 13 14 2004.18 Technopolis, et al. Using Logic models, The results of a study exploring how logic models can be used to develop a methodological framework for the high-quality assessment of IST-RTD effects at the Strategic Objective level, Commission Contract No 29000,2006. 19 W. W. Powell, in:
B. M. Staw, L. L. Cummints (Eds. Neither market nor hierarchy: network forms of organisation, Research in Organisational Behavior, vol. 12,1990, pp. 295 336.20 L. Blatter, Beyond hierarchies and networks:
a conceptual basis for uncertainty management in model-based decision support, Integrated Assessment 4 (1)( 2003) 5 17.8 G. S. Altshuller, Creativity as an Exact Science
developed patents in the fields of pricing and promotion algorithms, been a research fellow at the Technology policy Assessment Center of Georgia Tech,
Following a widely shared definition, foresight aims at improving future-oriented decision making through the early detection and assessment of emerging trends and drivers of change 18.
and its regional context 30.2) Other applications of scenario planning are framed as decision support instruments for the assessment of strategic options:
and specific trade-offs must be identified 32,33. 3) The outcome of an assessment of options in the context of different scenarios still depends on the type
and weighting of assessment criteria that are applied. In infrastructure sectors or other projects of public interest the range of goals is likely to be very broad
Others include the assessment of options but do embed not explicitly the planning in political decision making processes
used inadequate assessment methods and failed to conclude the process by actually informing strategic action 37.2.3.
and stakeholder preferences for the assessment and making impacts of system choices transparent. To tackle this claim in an adequate way,
a widening of participation in the assessment phase is necessary to include local knowledge for the establishment of regional specific scenarios 6 as well as for multi perspective assessment of options.
In terms of methodology, the value driven assessment follows the multi-criteria analysis approach (e g. 52) which is expanded to include uncertainties of context conditions (see e g. 53.
and the utilization of selected assessments'results. 3. The Regional infrastructure foresight method In view of this specific problem constellation and based on the theoretical arguments introduced above,
Fourthly, the assessment of the options follows the well established method of sustainability value tree analysis 66,67.
In the end, a list of assessments will be available for each option under each scenario and evaluated according to the preferences of each considered interest group. 3. 2. Exploring the landscape of trade-offs These results may then be analyzed via different perspectives.
we have developed two graphical representations of the assessment data. The first one relates social preference for each option in each scenario to its potential social conflict level (see Fig. 2). For simplicity's sake,
Options in the far right corner show strong differences in assessments among stakeholder groups. Hence, decision makers should be aware of potential divergence of interests.
because arguments for a more favorable assessment had been raised at least by one stakeholder group. Trade-offs related to context conditions and diverging value perspectives can now be represented in this diagram for each option.
The options'assessment took place in two steps. i) The core team assessed the options'generic strengths and weaknesses in reaching sustainability goals in a well balanced way for each of the different scenarios.
ii) In the second workshop, participants carried out the same assessments by taking the perspective of typical future citizens or industry representatives.
The desirability conflict diagram resulting from these assessments is presented in Fig. 4. The general picture shows that all three options are characterized by extensive polygons
and that consensus between the stakeholders is often lacking The onsite treatment option received highly diverse assessment scores.
In the downturn scenario, assessment results were rather antagonistic. In the high quality of life and the powerful region scenario, both groups ranked the solution as rather problematic or not desirable.
A comparison of the stakeholders'assessment with the sustainability evaluation of the core team offers insight into the sustainability deficits that a politically negotiated solution would entail (see Fig. 5)
Fig. 5. Sustainability deficit assessment in the Kiesental case study. 1159 E. Störmer et al.//Technological forecasting & Social Change 76 (2009) 1150 1162 The elaboration of a joint system representation by the different organizations in the Kiese catchment furthermore prepared the ground for intensified collaboration and shared visions.
We developed a specific methodological layout of a strategic planning process the Regional infrastructure foresight method that builds on a combination of foresight approaches that focus on exploratory context scenarios, option assessment and multiple perspectives.
Overview and Interpretative Framework, European Science and Technology observatory (ESTO), Paris, 2001.57 I. Miles, Appraisal of Alternative methods and Procedures for Producing Regional foresight, EU Kommission, Brüssel, 2002.58 R. Popper,
the increasingly complex world creates new types of risks that shouldn't be bypassed with the examination of future opportunities, creation of shared visions and assessment of desired impacts.
Popper 9 lists and describes 33 different foresight and assessment methods altogether. A tentative systemic framework of the potential FTA METHODS by Saritas 10 is shown in Fig. 2;‘
A generic model of the risk assessment procedure, applicable within the Nordic countries, will initially be framed.
Fig. 5 also shows that the assessment of new emerging risks demands longer time-frame and a more contingent examination approach.
That is why network building is the crucial part of assessment processes and methodologies. We can cope with the uncertainty by collecting
Industries Hazard Identification, Assessment and Control, vol. 1, 2nd ed. Butterworth-Heinemann, Oxford, 1996,357 pp. 15 E. Homberger, G. Reggiani, J
The first part sets out the respondent groups'assessments concerning the techno-scientific competence prospects and young people's interest in a number of professions.
The second part depicts the respondent groups'assessments of Finnish research activities, the prevailing state of technology development and various societal institutions,
The fourth part in the survey sets out assessments of sustainable development focusing on environmental threats the state of the environment,
Assessments of the younger citizens clearly point out that in the future science and technology will be followed increasingly through means of interactive,
Especially young people's assessments point out the fact that, in the future, science and technology will be followed increasingly by means of interactive,
a critical assessment. STI Working Paper Series 2005/3, OECD, Paris, 2005.3 M. Naumanen, Tekbaro Teknologiabarometri kansalaisten asenteista ja kansakunnan suuntautumisesta tietoon perustuvaan yhteiskuntaan
The second phase (see Fig. 2) of the searches ended with a first assessment of the topics that were found.
in order to get broader assessment on the topics, their importance and their time frame (see below.
Especially the online survey which had the function of focussing the topics as well as an assessment function is described in more detail (Fig. 3). 3. 1. Starting phase of the process The process started with 14 broader topic fields that were derived from the German High-tech Strategy 2,
The online survey is a contribution to the assessment phase, but cannot be a static selection mechanism for BMBF (see also 11).
For this, a last assessment by the international Monitoring Panel (second wave of partly standardised interviews) took place to validate the results.
For this, in a broad online survey, many participants are approached to involve more persons than the well-known lobbies in the definition and the assessment of new topics.
from the current point of view, there are already some contributions. 5. Preliminary assessment and outlook Until the end,
Strategic intelligence for an Innovative economy, Springer, Berlin, 2008.26 A g. Pereira, R. von Schomberg, S. Funtowicz, Foresight Knowledge Assessment, International Journal of Foresight and Innovation policy, vol
The slim evidence base aggravates the assessment, but the potential of scenario planning to prepare public policy-making for the uncertainties
5. Policy measure effectiveness assessment or policy termination, i e. to identify the effects of the policy measure
and weaknesses of several different exercises and assessments of the impacts scenario planning approaches can have on decision-making processes Fig. 1. Forms of scenario-based decision support. 1200 A. Volkery, T. Ribeiro/Technological forecasting
This argument is underpinned well by the study of global environmental assessments, which also emphasizes the need for salience,
whereas the assessment as such is carried out by the responsible directorate-generals of the Commission. These assessments are required to assess ex-ante several regulatory options.
Demanding a check of important decisions against a set of more alternative, long-term scenarios could help to broaden the focus of assessments,
which several evaluation studies found as one weakness of efforts so far 41. Another entry point could be Strategic Environmental Assessment.
An example can be found in Scotland, where scenario planning has been part of developing Scotland's 2nd National Planning Framework (NPF),
Does this assessment mean that future efforts should concentrate rather on indirect forms of scenario-based decision support,
as in the case of global environmental assessments such as the Millennium Ecosystem Assessment or the IPCC, work on the systematization of methods
Scenario development and Analysis for Forward-looking Ecosystem Assessments. In: MA Millennium Ecosystem Assessment (in prep. Ecosystems and Human Well-being:
A Manual for Assessment Practitioners, forthcoming. 7 B o'Neil, S. Pulver, S. Vandeveer, Y. Garb, Where next with global environmental scenarios?
Environ. Res. Lett. 3 (2008) 045012.8 P. Bishop, A. Hines, T. Collins, The current states of scenario development:
1207 16 P. Sabatier, H. J. Jenkins-Smith, The advocacy-coalition-framework. an assessment, in:
their development and use, Sub-report 2. 1b of Synthesis and Assessment Product 2. 1 by the U s. Climate change Science Program and the Subcommittee on Global Change Research, Department of energy
Lett. 3 (2008) 045015.22 W c. Clark, R. B. Mitchell, D. W. Cash, Evaluating the influence of global environmental assessments, in:
Global Environmental Assessments: Information and Influence, MIT Press, Cambridge, 2006.23 J. Oglivy, E. Smith, Mapping public and private scenario planning:
and New york. 41 J. Hertin, K. Jacob, A. Volkery, Policy appraisal, in: A. Jordan, A. Lenschow (Eds.
Assessment of Strategic Alternatives, Edinburgh, 2008.43 K. Van der Heijden, Can internally generated futures accelerate organisational learning?
His research interests are integrated environmental assessments, long-term policy analysis and institutional arrangements for effective scenario planning. He has authored
Ulrich provides a list of questions that can be used as a starting point for the assessment of the boundaries of the situation
and assessment tools for the analysis and improvement of the societal embedding of innovations. Socrobust emphasised the difference between hot unarticulated, open-ended (fluid/hot) situations and more structured and well articulated, stable (cold) situations 22.
Different types of participant have different assessment routines and practices, and one must acknowledge these bring them out in the scenarios
and create opportunities to become more reflexive of how the different participant groups make assessments.
and probe each other's assessment worlds. The bridging events can occur in an ad hoc way,
assessment (constant vigilance), and the need to interface promoters and selectors, arrive on governmental agendas.
but framed by self-assessment mechanisms (degree of precaution unclear). Innovation actor's quality not assured.
but framed by self-assessment mechanisms (i e. the actual degree of precaution is unclear) Tension:
this is part of ongoing assessment 33. However, as mentioned in Section 4. 2, these types of scenarios do stimulate discussions,
Analysis, participation and power in the social appraisal of technology, in: M. Leach, I. Scoones, B. Wynne (Eds.),
The paper gives a comprehensive review of the foresight and the assessment of its impacts on policy.
However as a result of the critical and systemic assessment of the process the*Corresponding author. Email: ozcan. saritas@mbs. ac. uk ISSN 0953-7325 print/ISSN 1465-3990 online 2009 Taylor & francis DOI:
New technology foresight, forecasting and assessment methods, 13 14 may, Seville, Spain. EU DG Research. 2005. Final Report.
The finer the criteria, the better the assessment at least in theory. But in practice, too many criteria make assessment a mammoth task
and few participaant will have the time (or patience) to devote to this. Furthermore, many participants will probably lack the requisite knowledge to make such detailed assessments.
This is an important (and often under-reported) limitation in many priority-setting exercises. In addition to the contribbutio a particular thematic area might make to scientific understanding, socioeconomic development, etc.
and depth of local expertiis to make prioritisation assessments. Arelated issue here is whether existing scientific communiitie alone are placed best to set S&t priorities.
5 but their full assessment was undertaken through an online questionnaire survey of much of the Luxembourg research landscape (around 300 participants), Online Questionnaire Young Res.
Workshop Exploratory Workshop‘Long list'of research domains Initial assessment Selection of broad research fields Stakeholder workshops No formal input General Challenges for Luxembourg Expert workshops per field SWOT Analysis Initial
assessment of domains Future trends Luxembourg context Research priorities candidates Conclusion & recommendations Phase 2: Setting Context/Identifying Priorities Data collection Bibliometrics Interviews International research trends Evaluation of FNR programmes Mapping of Lux.
It was at this level that respondents to the online survey were asked to make their‘attractiveness'and‘feasibility'assessments,
or another in the online survey (Phase 1) and in the thematic group workshops (Phase 2). This meant that expert assessments of domain areas against the criteria were readily available to draw upon.
'In this way, the‘new'assessment can be viewed as a‘sense-making'process that conferred ownership of foresight results to the MCHER.
This therefore made assessment against some of the criteria very difficult and probably unreliable. 15 7. 4. Who prioritises?
The assessment was deemed therefore to be an unreliable basis upon which to rank research domains. 11.
New technology Foresight, Forecasting and Assessment Methods, 13 14 may, Sevilla, Spain. Downloaded by University of Bucharest at 05:09 03 december 2014
a more critical and systematic assessment of the process shows that the apparent short-term success of the process seems to come at the cost of limited medium-to long-term impact, associated with a low degree of novelty and the avoidance of controversy.
thus facilitating the implementation of policies. 2. 3. An assessment framework for foresight processes in support of policy making Against this background,
Giving an assessment of longer-term impacts is difficult after just a year since the completion of the process.
< Back - Next >
Overtext Web Module V3.0 Alpha
Copyright Semantic-Knowledge, 1994-2011
