Quantitative and qualitative approaches in Future-oriented technology analysis (FTA: From combination to integration? Karel Haegeman a,, Elisabetta Marinelli b, Fabiana Scapolo c, Andrea Ricci d, Alexander Sokolov e a European commission, JRC-IPTS, Edificio Expo WTC, C/Inca Garcilaso 3, 41092, Sevilla, Spain b European commission, JRC-IPTS, Seville, Spain c European commission, JRC, Brussels, Belgium d ISIS, Rome, Italy e Institute for Statistical Studies and Economics of Knowledge, Higher School of economics, Moscow, Russia a r t i c l e i n f o a b s t r a c t Article history: Received 14 may 2011 Received in revised form 9 july 2012 Accepted 3 september 2012 Available online 8 november 2012 The FTA COMMUNITY relies on a set of disciplines and methods, which try to better understand and shape the future from different methodological perspectives. Whilst the community has grown since the first edition of the International Seville Conference on Future-oriented technology analysis (FTA), there is still little dialogue and exchange between those applying quantitative and those applying qualitative methods. The FTA events have, since the beginning, provided an avenue to debate methodological aspects and this paper summarises and furthers the discussion developed during the 2011 edition, building on the debates at the conference and between members of the conference Scientific Committee, to which the authors of this paper belong. In particular this paper describes the methodological state of the field through a tripartite taxonomy of increasing levels of qualitative and quantitative integration. It shows how significant progress has been made for simpler forms of combinations but not for more sophisticated (and perhaps more promising) ones. Following that, it suggests that an epistemological divide, common to the social sciences as a whole, combined with cultural differences and misconceptions within the FTA COMMUNITY are amongst the factors undermining further methodological integration. The paper concludes by suggesting some steps combining research and practice, to overcome such barriers. 2012 Elsevier Inc. All rights reserved. Keywords: Qualitative Quantitative Barriers Combination Integration FTA Epistemological divide 1. Introduction The methodological debate has been a relevant element of the International Seville Conference series on Future-oriented technology analysis (FTA ) since its first edition in 2004. For the 2011 edition, the Scientific Committee decided to focus specifically on the combination of quantitative and qualitative methodologies. This article, drawing on a position paper written for the conference, reports and expands on the lessons learnt during the event. In order to define the scope of our paper it is important to first clarify what we mean by quantitative, qualitative and participatory FTA METHODS. We define data as quantitative when consisting of numerical information and a methodology as quantitative when applying statistical/mathematical tools. In contrast, we define data as qualitative when consisting of non-numerical information (such as text, images, and sounds) and a methodology as qualitative when not relying on statistical/Technological forecasting & Social Change 80 (2013) 386 397 The views expressed are purely those of the authors and may not in any circumstances be regarded as stating an official position of the European commission. Corresponding author. Tel.:++34 95 44 88 392; fax:++34 95 44 88 326. E-mail addresses: karel-herman. haegeman@ec. europa. eu (K. Haegeman), elisabetta. marinelli@ec. europa. eu (E. Marinelli), fabiana. scapolo@ec. europa. eu (F. Scapolo), aricci@isis-it. com (A. Ricci), sokolov@hse. ru (A. Sokolov. 0040-1625/$ see front matter 2012 Elsevier Inc. All rights reserved. http://dx. doi. org/10.1016/j. techfore. 2012.10.002 Contents lists available at Sciverse Sciencedirect Technological forecasting & Social Change mathematical tools. A participatory method, regardless of the qualitative or quantitative data it uses, is one in which the outcome requires the active interaction of different types of stakeholders. It is acknowledged that, within the FTA COMMUNITY (which comprises Foresight, Forecasting and Technology assessment), 1 foresight practitioners have concentrated traditionally on participatory methods based on qualitative data, on the grounds that quantitative extrapolation from past data is not sufficient to address the uncertainties of the future and that emerging changes in the socioeconomic and technological landscapes need to be taken into account. Another part of the FTA COMMUNITY, constituted by Forecasting and Technology assessment practitioners, holds an opposite standpoint, considering qualitative and participatory approaches as a second best option, to which we are compelled somehow to refer until adequate quantitative methods arise. Whilst these two branches of the FTA COMMUNITY operate largely separately, in the social sciences the mixing-methods debate has advanced considerably 2 6 and is illustrated well by the existence of dedicated publication outlets such as the Journal of Mixed Methods. Nevertheless in the discussions at the 2011 FTA Conference some trends were identified suggesting that methodological combination may potentially become more common amongst FTA scholars and practitioners. Just to mention a few key developments: the fast progress of the internet offers new opportunities for participatory projects, of both quantitative2 and qualitative nature; at the same time, the improved availability of S&t and innovation indicators and the advances in quantitative methods provide more input for quantitative analysis; finally, given the increasing stress on robust evidence for policy, accessing and combining different types of information and methods can only but better support policy-makers in their job. These tendencies raise important questions on the value and scope of combining methods, as well as on the reasons why, such combination, within the FTA COMMUNITY, seems to proceed at a lower rate than in other fields. This paper aims at reflecting on those questions. Section 2 looks at the potential value of using quantitative and qualitative approaches. Building on this, Section 3 proposes a taxonomy of methodological combinations at different levels of integration which is used then as a framework for classifying current FTA practices. Section 4 reflects on the obstacles that may be preventing a faster or deeper methodological integration and Section 5 discusses whether and how some of them could/should be eased. Section 6 summarises and concludes. It is important to stress that the reflections in this paper are elaborated on the basis of discussions at the FTA conference and the debate (both before and after the event) between members of the Scientific Committee, to which the authors of this paper belong. As such the paper aims to be a stimulus to further dialogue rather than providing an empirical stocktaking or a systematic and comprehensive review of literature. In addition although the reflections in this paper originate from inside the FTA COMMUNITY, they are addressed to a broader audience, based on the awareness that other branches of social sciences, which have so far not been in touch with the community, could also become more connected to FTA. It is in this sense an invitation to other communities to join and share their views on this topic at future FTA conferences. 2. Sense or nonsense of applying qualitative and quantitative approaches in FTA In social sciences alternative methods may encourage or allow expression of different facets of knowledge or experience 2, or even tap different domains of knowing 8. A detailed discussion on the reasons for applying a combination of methods in social sciences goes beyond the scope of this paper, but the reader is referred to Rossman and Wilson 9 and Caracelli and Greene 10. In FTA one of the main arguments for more use of quantitative approaches may come from policy-makers, who, in search for effective policies, tend to aim at reducing or at least understanding the uncertainties that can alter the outcomes of their policies and try to quantify them ex ante. In addition numbers are perceived as easy to communicate to a wider audience. However, one must be aware that the relevance of quantitative methods may lie more in their systematic process of comparing policy alternatives under different scenarios, than in the absolute values of their results. A further point to keep in mind is that numbers may create the impression that the future is less uncertain than it actually is. 3 In this respect Linstone 12,13 points out that the possibilities of forecasting may be limited especially when the dynamic, complex and adaptive nature of the systems we are dealing with today are moving from one technological era to another. Along similar lines Tuomi 14 suggests that the ontological unpredictability4 of innovative process cannot be removed by more accurate data or incremental improvements in existing predictive models. These observations lead to ask whether foresight performs better than quantitative approaches in supporting the design of effective policies. The answer is, as one would expect, not clear. Calof and Smith 15 have looked at what makes foresight successful. Literature has paid however little systematic attention to evaluating the actual impact of foresight 16,17. In the call for papers, the 2011 FTA Scientific Committee argued that the exclusive use of qualitative methods can lead to partial views on possible futures, e g. when relying on qualitative scenarios in devising technology roadmaps. They also identified the scientific quality and validity of outcomes of participatory approaches as an issue of concern 18. In addition, several authors indicate that foresight reproduces or projects current knowledge and assumptions 19,20 and does not tell us much about the future. Similarly, Loveridge and Saritas 20 point out that we simply cannot know the future. They see FTA (both quantitative and 1 The interested reader is referred to Scapolo and Cahill 1 for further details on the origin and definition of the acronym FTA. 2 Quantitative participatory methods could for instance relate to the online sharing of big amounts of data, the development of online models accessible to a whole community, or the engagement of a wider group of participants in data analysis (for the latter, see Cooke and Buckley 7). 3 In this regard, Goodwin 11 points to the effect of hindsight bias (the tendency to believe that our forecasts were more accurate than they actually were) on forecasting, making us believe that the world is a much more predictable place than it really is. 4 With ontological unpredictability Tuomi refers to the theoretical incompatibility between innovation and predictive models when disruptive and downstream innovation become more frequent, based on the argument that we can only retrospectively know what we are talking about, due to the unpredictability of natural, behavioural and social processes that shape innovation. 387 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 qualitative) as an imaginative projection of current knowledge in which formal methods and techniques play a subsidiary role (p. 753. Unsurprisingly, several authors plea for a stronger emphasis on creativity and exploration in future-oriented technology analysis, in order to identify truly alternative future developments and to enhance our capacity to live in an unpredictable world 14,20, 21. Both quantitative and qualitative approaches can contribute to this exploration, as explained by Eerola and Miles 22, who see FTA EXERCISES as attempts to collect knowledge about‘posits'or possible futures, their plausibility and limits, their internal consistency and conformity with models and data, their consistency with expert judgement, and their implications for action. A wide variety of information about the past and the present can contribute to this and what is crucial is to understand the opportunities that different methodologies can offer. In order to better understand those opportunities, we look in the next section at some current FTA practices. 3. A taxonomy of methodological combinations as a basis for identifying current FTA practices Although there have been many efforts on classification of methods and on the possible combination of qualitative and quantitative approaches 22 25 the literature provides little evidence on systematic comparison and analysis on the consequences and outcomes of applying specific techniques in the course of FTA. 5 Scapolo and Porter 27 argue that this absence of stocktaking analysis is mirrored also in the lack of guidance on how to evaluate FTA projects that combine different methodologies. Yet, evaluation on methods is important in order to identify‘good practices 'and to contribute to the robustness of FTA, in terms of process, content, and impacts. Since its first edition the International Seville Conference on Future-oriented technology analysis devoted part of its attention to the development of new tools and methods, novel use of existing methods and (new) disciplines applied by FTA. Despite the lack of systematic investigation, there are clear signs of interest from the FTA COMMUNITY in combining methods. There are obviously many ways to look at current practices. An interesting one is to group FTA EXERCISES based on the level of integration of qualitative and quantitative methods. We therefore propose a taxonomy of methodological combinations at three different levels of interaction and use this as a basis for looking at current practices in combining qualitative and quantitative FTA APPROACHES. 3. 1. Independent (parallel or sequential) use of qualitative and quantitative approaches In this type of FTA exercise qualitative and quantitative approaches are applied in parallel or sequentially, but separately from each other. Typically, when different methods/tools are applied within the same project, links consist mainly of using the results of one part as an input into another part. In some cases, the interactions between the two approaches are limited to cross-checking of assumptions and findings of the same analysis conducted in parallel with a quantitative and a qualitative approach. Examples of this type of combined use are quite common, and most projects in this category tend to have a rather straightforward methodological design. A drawback of this type of combination is that, whilst allowing in a relatively immediate way for multiple perspectives, it does not enhance cross-disciplinary learning between project participants, which is a suboptimal use of available FTA APPROACHES. Examples of current and upcoming FTA practices Output of qualitative approaches as input to quantitative approaches New indicators: Qualitative data can provide additional evidence to quantitative models by inclusion of new indicators created from quantified expert judgments. Cunningham and van der Lei 28 use such an approach for models providing support to decision-making on the selection of new technologies and discuss the issue of providing equilibrium between different groups of experts and stakeholders. Agami et al. 29 show the use of fuzzy logic for trend impact analysis combining surprise-free forecasts and consensus-based experts'judgments. Quantification of qualitative scenarios: Quantitative methods can also be used to process qualitative judgments for scenario design 30,31. Valette 32 points at opportunities for foresight exercises that combine expert-based contrasted socioeconomic and policy scenarios (qualitative part) and a mathematical quantification of the impacts of the alternative scenarios (quantitative part)( p. 239. Output of quantitative approaches that feeds into qualitative FTA Use of bibliometric and patent analysis as an input for further qualitative FTA: For instance, quantitative analysis of this kind offers valuable information for the development of S&t Delphi survey topics, or for the identification of potential disruptive technologies 5 10 years ahead. Shibata et al. 33 distinguish between incremental and radical innovations via 5 The European foresight monitoring Network is one of the few attempts to take stock of quantitative and qualitative foresight methods. The mapping identified only three quantitative methods (bibliometrics, modelling and simulation, trend extrapolation), highlighting that they were combined with literature review, scenarios and expert panels. These conclusions have to be interpreted with care, as there are sources of bias in the EFMN database (see Keenan and Popper 26 for a critical assessment of the exercise). Remarkably, evidence stemming from the forecasting communities on cases combining qualitative and quantitative methods is limited rather, suggesting that the need for combining methods is less felt than in the foresight communities, possibly because foresight is more recent and still wants to prove its usefulness. 388 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 identification of emerging clusters analysing citations and keywords for a particular technology field, which creates a background for further qualitative studies. Järvanpää et al. 34 analyse the use of bibliometric data for distinguishing between science-based and conventional technologies, giving experts an opportunity to focus on particular technology areas using relevant qualitative methods. Thorleuchter et al. 35 demonstrate that patent-based quantitative approaches to cross-impact analysis for the identification of relationships between technologies can be used instead of, or in combination with, traditional qualitative methods based on literature reviews. Identification of trends and wildcards: Quantitative methods can also be used to identify outliers (outstanding observations) which could be revised further by experts as potential wild cards. Visualisation of quantitative data 36 can be a useful way of bringing these data to a workshop or another qualitative process. It can provide experts with a powerful instrument to identify hidden trends, to monitor the convergence of adjacent technologies 37, or to identify Emerging s&t areas 38. Comparison of outcomes of qualitative and quantitative approaches Participants at the 2011 International Seville Conference on FTA raised the potential of the use of qualitative and quantitative methods for identifying and anticipating disruptive innovation, forecasting radical changes and thus limiting the scale of failures (with a focus on market pull vis-à-vis the technology push approach). In the NEEDS project (www. needs-project. org), the acceptability of future energy technology options was submitted to a multi-criteria assessment involving a panel of stakeholders, the results of which were compared to the findings of a quantitative calculation of performance indicators. 3. 2. Use of technical and methodological interfaces that facilitate interaction A second type of exercise uses intermediaries as interfaces between the two approaches. Those intermediaries can be (ICT) tools, or disciplines that are traditionally not related to FTA and function as an integrator between qualitative and quantitative approaches. For example, web 2. 0 tools allow for the collection of both quantitative and qualitative data or for the quantitative analysis of qualitative data (such as statistical analysis of stakeholder opinions or networking behaviour. Such exercises push experts in quantitative and qualitative techniques closer to each other, therefore enhancing cross-disciplinary learning. This type of learning is, however, static in that it is limited to the way the exercise is structured and does not evolve as the exercise progresses. Examples of this type of exercises are not many and are mostly in a development stage, they are nonetheless highly promising and it is therefore important to monitor their development. Examples of current and upcoming FTA practices Internet-based tools allowing for integration of data of various sorts Online sharing of perspectives on different data types: new technologies such as web 2. 0 can be used by FTA to streamline operations by increasing interactive participation of stakeholders, speeding-up the provision of information and feedbacks and integrating data of different sorts (pictures , documents, numerical data, free text, videos. 6 The Risk assessment and Horizon scanning Initiative (Singapore) developed a Service Oriented Based Horizon scanning Architecture (SOSA) allowing sharing perspectives on data sets in order to amplify data outliers and help users avoid getting blind-sided through premature convergence 40. It consists of an intranet based network of people, tools and data (from unstructured text from internet to reports uploaded by experts), and the sharing of perspectives across the network is supported by a set of perspective visualisation tools. Online analysis of data and creation of knowledge repositories: Cooke and Buckley 7 believe that web 2. 0 tools can be used to make data of all sorts accessible to respondents and researchers: Respondents no longer merely respond to signals: they generate the data, they edit it, via their communal participation, revising it in response to others, irrespective of whether the others are researchers, clients or respondents (p. 289). However, to date no concrete examples of this approach could be identified, though the authors refer to Wikipedia and Google as similar examples. A related approach is proposed by Soojung-Kim Pang 41 who suggests that tools and methods of social scanning and prediction markets could be used to improve professional forecasting and foresight in an era of complex phenomena and disruptive events with high level of uncertainties. Making scanning results public would allow people concerned with complex phenomena, like geopolitics, climate change, or issues at the interplay between science and society, to keep track of the content of one's intellectual portfolio of knowledge and to aggregate results into a broad view of what the future might entail. 6 Yet, the use of collaborative tools such as social web platforms has been limited rather, both in FTA practices and in linking policy and research in general (for an example, see Haegeman et al. 39). 389 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 Other tools and disciplines that can serve as interface to facilitate the use of qualitative and quantitative approaches and data Social network analysis: Social network analysis has attracted attention in the past years allowing quantitative analysis on relationships and links that make up various social processes. When used in combination with foresight data collected online, network analysis can be used to enable robust analysis of foresight data, which are often complex to present and codify. Nugroho and Saritas 42 propose a framework for this, building on online foresight survey data, and by pointing at benefits in the various phases of a foresight process: it reveals the structural features of the data and can inform the foresight process on emerging links or relationships, groups or clusters. The implication for foresight methods is that network analysis can introduce a‘systemic'perspective emphasising relationships between actors, key issues and trends. Visualisation techniques and strategic design: During the 2011 International Seville Conference on FTA, the use of images and visualisation techniques was suggested as a tool, both for better explaining (quantitative and qualitative) methodologies adopted and for results obtained. A method that makes use of visualisation techniques is the application of strategic design in addressing societal challenges. It explores and visualises‘the architecture of problems 'and can offer a wider framework in which quantitative and qualitative approaches could contribute to addressing the same problem. Further exploring synergies between FTA and strategic design could offer opportunities for more integrated use of methodologies. 7 3. 3. Full integration of qualitative and quantitative FTA A third type of exercise goes beyond the use of interfaces, and relates to an integration of qualitative and quantitative approaches at all stages of the FTA PROCESS. In this case results of qualitative and quantitative techniques feed into each other at several stages of the process or pass through several iterations. This is expected to generate dynamic cross-methodological learning processes as at each phase or iteration of the exercise ideas flow between different domains of knowledge. Examples of such deep integration are scarce, not only within the FTA COMMUNITY, but more generally also in social science where most reports of mixed methods studies report either parallel or sequential component designs 2. Nevertheless, the authors believe that such deep integration of methods can potentially enhance both the analytical depth and the policy impact of fta activities. Examples of current FTA practices Examples of research with deep integration of qualitative and quantitative FTA APPROACHES are not many. One example constitutes the Millennium Ecosystem Assessment scenarios, linking ecosystem change and human well-being by combining qualitative storyline development and quantitative modelling through several iterations between both parts 44. In the scenario development it was possible to identify areas where analytical tools are relatively weak and can be improved. In the quantification of ecosystem service scenarios it was possible to identify where advancements are needed in terms of model developments. 4. Barriers to integrating FTA METHODS The above taxonomy and the (non exhaustive) review of current practices show that various attempts have been undertaken to combine quantitative and qualitative approaches. It also raises the question as to why simpler forms of methodological combination are developed relatively well whereas more complex ones are much less common. This section elaborates on these aspects by discussing some of the barriers that may hamper a deeper integration of quantitative and qualitative FTA METHODS. 4. 1. Barriers stemming from an epistemological divide in social sciences The main barrier to a full methodological integration is epistemological in nature: there is a longstanding debate in social sciences (which is confined not to the FTA COMMUNITY) on the type of knowledge that qualitative and quantitative methodologies can produce and on the value of combining them 45 49. Suffice to say that there is a strong opposition between those who argue that quantitative and qualitative approaches are incompatible 50,51 and those who argue the opposite 52 57. Despite these and other longstanding attempts 58 61 for methodological combinations, the combination of quantitative and qualitative methods is still not the norm, indicating that the lack of quantitative and qualitative integration in FTA stems from more fundamental barriers outside the community. For example even in projects combining quantitative and qualitative methods, data 7 At first sight, this method is more suitable for FTA for businesses. However, it has been applied by the Finnish innovation Fund to develop holistic understanding of a challenge with issues related to ageing, education, and sustainability 43 (See http://www. sitra. fi/en/articles/2012/strategic-design-finlandsneewapproach-problem-solving). 390 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 are brought not always together in the analysis 62 and qualitative and quantitative tasks are carried out by different teams, with some kind of bridging mechanism to connect the respective outcomes. 4. 2. Cultural differences in FTA A layer of complexity is added to this methodological debate in the case of FTA, where the future is the object of analysis, as there are different ways of exploring the future, based on e g. differences in beliefs or educational backgrounds. Still today the different FTA COMMUNITIES are perceived as rather competing than collaborating in exploring the future. In the context of qualitative and quantitative techniques especially the marrying of quantitative modelling and foresight seems to be unexplored rather. The idea that one can forecast or predict the future seems to be contradicting the idea of developing multiple futures and shaping the future by identifying common directions to follow, using foresight. In reality, predicting certain elements of a broader system such as demographic developments is not in contradiction in any way with developing multiple futures. In Section 2 we pointed to the need to understand the opportunities that different methodologies can offer in collecting knowledge about possible futures 22. However, understanding these opportunities does not seem straightforward. The shortage of examples of full methodological integration (as shown in Section 3 may also reflect deeply ingrained cultural differences hindering good communication between different epistemological communities. Efforts to address this cultural clash have focused often on the adaptation of the methods and tools commonly used by the two communities, in the hope that adequate interfaces could be found facilitating operational collaboration. However, one could paradoxically observe that the emergence of structured interfaces between narratives and numbers has consolidated the historical practice of both communities working in isolation: each can safely remain in their own cultural realm, as long as a mechanism can be identified to ensure some level of communication 63.4.3. Misconceptions within the FTA COMMUNITY In order to better understand the pros and cons of different methodologies, three implicit misconceptions within the FTA COMMUNITY need to be addressed. Firstly, it is assumed often that models belong exclusively to the quantitative domain and have objective predictive power. Models are simplified a representation of reality and can be quantitative or qualitative, depending on the type of data they rely on. They may generate as an output, informed estimates about the future. The latter, in the case of quantitative models, take the form of numbers with associated probability distributions and confidence intervals (depending on the model). However, in the case of long-term horizons, such informed estimates have limited only predictive value. As clarified by Lüdeke 64, the value of models is not so much in their ability to tell us with a degree of certainty what will happen to society, but rather in their ability to structure thinking on the basis of the information available. Therefore, a model is more valuable as an analytical rather than a predictive tool. This means that both quantitative and qualitative tools and techniques should be judged not so much against the accuracy of their prediction on the future but against the assumptions upon which they rely, the data (advantages and limitations) that have been used, and the alternatives (or lack thereof) amongst which the analyst had to choose. In other words, when dealing with issues surrounded by risk, uncertainty, and ignorance typically issues being dealt with by FTA the value of models is (at least) as much in the process as in the output. 8 Another common misconception associates subjectivity and value judgement to qualitative processes, but not to quantitative ones. Scientists (particularly natural scientists and technologists) often tend to consider subjectivity, e g. experts opinions as a disturbance to be avoided, or at least framed and assessed, in order to reduce its distorting effects on the interpretation of evidence and its decision-making powers. There are two weaknesses in this claim. First, regardless of the quantitative or qualitative nature of the approach followed, a value judgement is involved in any FTA exercise. The distortion of such subjectivity can be reduced by the legitimacy of the persons making the judgement (whether it is a modeller choosing a functional form or an indicator, or a foresight practitioner identifying the right stakeholders to invite to a workshop). Second, when FTA and especially foresight is addressing the intrinsically uncertain challenge of devising possible futures, subjectivity, meant as expert/stakeholder opinion, is effectively a source of creativity and as such should be exploited fully. In general, foresight aims also to envisage a wide and contrasted set of options that can be devised by eliciting experts'knowledge. Whether one or the other of such options will eventually materialise depends, amongst others, on the actions of those who will forge the future, largely reflecting their individual and collective (evolving) preferences. Capturing the subjectivity of FTA contributors, i e. their valuable yet tacit (not codified) knowledge, is therefore of the essence. 9 8 During the 2011 FTA Conference a lively discussion was devoted to the shift of FTA usage from exploring potential risks to inspiring sustainable innovation. In this respect taking into account expert opinions is also important for the identification of key clusters and addressing different types of innovation. 9 A good example is the contribution that FTA can provide to policy and decision makers in charge of the prioritisation of alternative technological options. The long term prospects of emerging technologies are usually scrutinised through the lens of e g. Cost Benefit Analysis or similar tools that require the valuation (or at least some form of quali-quantitative estimation) of a variety of factors including the social costs and benefits that can be expected from the diffusion of the technologies under scrutiny (how will their diffusion affect quality of life, the conservation of natural resources, landscape integrity, ecosystem services etc.).In the absence of market values for such goods and services, valuation practices commonly recur to Contingent Valuation Methods (CVM), which aim at eliciting information on the hypothetical dynamics of future preferences (individual and collective). The appraisal of the expected future performance (and the ranking) of alternative technological options therefore explicitly incorporates information that inherently reflects the subjectivity of social players, and its value and credibility are enhanced thus. 391 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 Finally, in this debate, there is a tendency to equate qualitative with participatory. This is clearly inaccurate, as participatory methods can be (and in fact are adopted in the framework of quantitative analyses as well. On the other hand, qualitative approaches have been adopted for many decades (e g. scenarios) with no other involvement than that of the FTA EXPERTS. Acknowledging these aspects is essential to identify a common ground for methodological integration. 4. 4. Lack of skills and trust The aforementioned epistemological barriers are reflected in, and reinforced by, the lack of researchers, practitioners and evaluators skilled in both quantitative and qualitative FTA APPROACHES. 10 It is for instance neither common, nor easy, for those who come from a strongly quantitative perspective to communicate the core of their work to anyone outside their community. This lack of skills to understand and interpret both approaches leads to lack of trust amongst practitioners of each approach. Such lack of trust again reinforces existing epistemological barriers, as illustrated in Fig. 1. This link between lack of skills and lack of trust occurs not only within the FTA COMMUNITY, but also amongst its clients (in particular policy-makers. Trust from policy-makers in outcomes from an FTA exercise may be undermined by differences between the expectations of decision-makers who should ultimately use the outcomes of FTA, and what FTA can really deliver. For example, Georghiou and Harper 66 point at the concern that administrations may want to harness FTA in order to constrain uncertainty to the point where traditional tools may be used. This is however not what the FTA COMMUNITY is set out to provide. Similarly, decision-makers can claim that the wrong people were involved in a Delphi survey, or they simply distrust the foresight process, because they have not been involved themselves 67. Finally, policy-makers are concerned still very much with the shorter term, which may have its influence on (lack of) trust. 11 5. Overcoming barriers: short-term/long-term As the examples provided in Sections 2 and 3 suggest that methodological integration can be promising and beneficial to the field, it seems important to reflect further on the aforementioned barriers and on potential actions to overcome them. Below we propose a dual approach, combining the short-term treatment of‘symptomatic'barriers, with longer-term priorities for research and policy that address more fundamental barriers to methodological integration. 5. 1. Overcoming short-term barriers One way to extend the application of quantitative methods in FTA is their gradual integration into existing practices of FTA and their convergence with qualitative techniques. Addressing practical incompatibilities between both approaches may also gradually change preferences of FTA practitioners for specific methods in favour of integrated approaches. Below we sketch a possible agenda to overcome the main short-term barriers regarding methods selection and misconceptions. 5. 1. 1. Methods selection The selection of methods in FTA remains largely a context-driven issue, as there are no‘recipe books 'and only limited attempts12 to better clarify the relation between context, content and approach of an FTA project 68 70. This is even more the case when different types of methods are applied where it is important to combine or integrate both approaches or identify interfaces, making use of complementary and/or contradictory features of methods. Given the lack of identified good practices as a natural first step to tackle this issue, one could consider developing a starter kit for combining different methodologies. In social 10 The lack of human capital skilled in both quantitative and qualitative methods, could be traced potentially back to the education system, where students are confronted early with choices between different options (such as the divide between social and natural sciences), which often implicitly entails choosing for a more quantitative or qualitative training. In this regard Reiss 65 and Bryman 62 refer to the need for overcoming‘trained incapacities'.'11 Interestingly, our reflection on the epistemology skills trust triangle is in line with Bryman 62 who focuses on the integration of qualitative and quantitative data and methods in social sciences. Based on interviews with social scientists he identifies eight barriers to integration of qualitative and quantitative data and methods, which can also be applied in the field of FTA: perceptions on the expectations of different audiences, methodological preferences of the (mixed methods) researcher, structure of the research project, different timelines for different method types, skill specialisms, the nature of the data, ontological differences, tendency of some journals to emphasise one type of research, and lack of well-known good practices. Many of those seem to be somehow related to the epistemology skills trust triangle. 12 One could also look into project design papers, consultancy reports, and other types of grey literature in order to advance knowledge in this area. Future research could take up this endeavour. Fig. 1. Epistemology-skills-trust cycle. 392 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 sciences, Cameron 71 developed the Five Ps Framework, 13 which provides a mixed-methods starter kit, including a checklist of contentious issues, risks and traps that require consideration. It could be adapted to serve as an FTA starter kit for combining methods. In addition, a short-term research agenda could help addressing practical incompatibilities that hamper the combination of different methods. A possible item on such research agenda could be screening specific sectors where quantitative FTA APPROACHES are represented traditionally stronger, in view of identifying good practices. Such sectors include energy 32 (see the Energy Foresight Network: www. efonet. org), transport and climate change. Also in regional cohesion policy there are possibilities for blending quantitative and quantitative approaches, specifically in ex-ante (and ex-post) impact assessment. 5. 1. 2. Misconceptions In order to avoid getting stuck in the circle of epistemological barriers, lack of skills and lack of trust, misuse of methods should be avoided and transparency increased. In particular, from the start of an FTA PROCESS, it is necessary to clarify the definition of terms used and the basic set of assumptions, on which the exercise (in its quantitative and qualitative components) rests. In this way it is possible to make more intelligible how data are collected processed and analysed in the process. Unavoidably, policy-makers and stakeholders will assign a higher plausibility to scenarios that somehow resonate with their own visions. On the other hand, policy-makers and stakeholders will contribute to shaping the future through their decisions and their actions. As a result, their direct involvement in the foresight process will not only increase its robustness, it may also lead to higher acceptance levels and legitimacy. A good management of stakeholders'participation, is therefore critical to improve trust within the community. In particular, stakeholders'participation should reflect the overall need of a finely tailored balance between quantitative and qualitative dimensions and to this end directly involve forward-looking experts from both communities. Ultimately, building on and extrapolating from Wehnert and Jörß 67 we believe that participation of stakeholders and practitioners from different methodological traditions is crucial at least at three major stages of FTA PROCESSES: The initial identification of issues is one of the stages where the involvement of stakeholders is essential as it allows collecting the views of those who will play an active role in the decision-making process supported by FTA. During this stage, it is recommended also to include the wider range of stakeholders that will be affected by the consequences of these decisions. The exploration of future possibilities, when the expertise and knowledge made available by stakeholders and experts in the specific sector (s) subject to FTA can contribute to significantly increasing the quality of FTA results. The discussion and interpretation of results (including devising recommendations when sharing FTA findings with stakeholders who are expected to follow up with action are essential for building ownership whilst providing decisionmakker with insights on the limits of the FTA results. Such a stepwise organisation of the participatory process allows for intermediate validation milestones, thus also decreasing the risk of stakeholders dominating the FTA PROCESS. 14 The importance of this in the context of this paper lies in keeping a balance in participation of stakeholders and practitioners from different methodological traditions at each stage. 5. 2. Overcoming long-term barriers Overcoming barriers such as the lack of identified good practices, lack of trust, and differences in cultural backgrounds requires a longer time horizon and below we suggest some of the steps that could be taken in such direction. 5. 2. 1. Lack of identified good practices In light of the growing volume and variety of FTA practices, a stocktaking of basic knowledge (including e g. relevant statistical evidence, existing scenarios and corresponding outputs, coherent assumption sets) could be beneficial. Such stocktaking endeavour should compare FTA EXERCISES according to a set of variables (such as goals, methodologies, time horizon15) and provide both methodological and evaluation guidelines. The experience of the European foresight Platform (www. foresight-platform. eu) could provide some relevant inputs towards this endeavour. As previously noted these methodological and evaluation guidelines should explicitly account for the wide range of context-dependent choices, and provide accordingly differentiated support to FTA practitioners. They should also pay particular attention to validity and uncertainty of its main subjects. Validity should be measured by checking whether the cause effect mechanisms at play are represented in a credible and transparent manner, and not in terms of the probability that an envisioned future eventually comes true. In turn, uncertainty should be assessedwith the primary aim of differentiating between the intrinsic variability of a given phenomenon that exhibits high sensitivity to small changes (e g. networks congestion) and the uncertainty that derives from an insufficient knowledge of complex phenomena (e g. climate change). Ultimately, what matters is that methods and tools provide as far as possible a 13 The 5 P's Framework includes Paradigms, Pragmatism, Praxis, Proficiency and Publishing. 14 Da Costa et al. 72 illustrate the stepwise organisation of the participatory process by suggesting the application of varying degrees of participation in each phase of the foresight process and thus tailoring foresight phases to different foresight functions. 15 Typically, quantitative models present higher credibility for shorter time horizons, and qualitative approaches with storylines for longer ones. An exception is the International Futures Model, which can be used to examine long-term and interacting global development issues 73.393 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 reasonable representation of the systems being analysed, and that the intrinsic uncertainties associated with such representation are documented at best. 5. 2. 2. Lack of trust One aspect of trust is that it derives from perceived credibility, which is intrinsically problematic when referred to long-term, unverifiable representations of possible futures. Past performance is considered commonly as the most useful indication on which to build credibility and trust. For forecast exercises, past performance can be validated by comparing predictions to reality. 16 Foresight on the other hand does not claim to predict the future and therefore checking if the most favourable, recommended future state has been realised is not meaningful. Good practices should rather be sought by measuring, if at all possible, the benefits derived from the foresight process itself (such as for instance network building and improved collaboration) and the improvement of the decision-making process, or in assessing whether organisations (including private companies) that consistently adopt foresight approaches, perform ultimately better than those that do not. Obviously, such endeavour would require the existence of both good performance indicators and the identification of correlations and causal relationships between the scores on the indicators and the anticipatory approaches applied. Some recent work 15 17,74 moves a step closer into the direction of assessing the impact of foresight, but a long way is still to go. Much can also be learnt from the experience that the field of foresight has developed in dialogue and communication in support of building trust amongst stakeholders. For instance, the consultative dimension of SEA (Strategic Environmental Assessment) has proven essential to build consensus amongst stakeholders around the long-term effects of large infrastructure projects. Techniques and approaches related to building stakeholder trust can also be applied to bridge qualitative and quantitative communities. One important element here is the use of a common language or, ensuring that concepts and definitions are understood well and translated between communities. Increasing trust between communities can also be pursued, from a strictly methodological perspective, by increasing the mutual understanding of the approaches applied. Stocktaking of good practices, and the development of frameworks that support the selection and implementation of an appropriate combination of qualitative and quantitative methods and tools in a given context, can enhance shared knowledge, which may in turn increase trust. However, increasing mutual understanding, whilst always desirable, may not always be possible, either because the cultural differences are rooted deeply, or because the language used is intrinsically different. Still in such cases, combinations are possible. White 75 suggests that combining methods mainly needs mutual professional respect. Looking at collaboration in the wider context of collaborative governance, Ansell and Gash 76 conclude that collaboration between stakeholders tends to develop when it focuses on small wins that deepen trust, commitment, and shared understanding. This pathway of small wins may also have potential for increasing trust, commitment and shared understanding between methodologically different FTA COMMUNITIES, which again indicates the importance of stocktaking on good practices. 5. 2. 3. Differences in cultural backgrounds: closing the epistemological gap In the long run the clash of cultures could be transformed into an asset by developing the right skills to improve the community's work, the level of trust and, ultimately, the policy impact of fta. To address the epistemological barriers and gain legitimacy and trust it is important to establish links and collaborations between the various sub-communities. These should aim at understanding the reasons behind differences in methodological preferences, and at reducing the barriers between quantitative and qualitative training. In so doing, due importance should be given to developing the right communication skills for sharing ideas and for describing the processes and outcomes of both qualitative and quantitative work, within and beyond the FTA COMMUNITIES. 6. Conclusions and further steps Although systematic integration of quantitative and qualitative methods in FTA is not a standard practice so far, evidence in this paper has shown practical examples of combinations of these types of methods. The different and highly heterogeneous contributions to the 2011 International Seville Conference on Future-oriented technology analysis in this area share a common bottom line: both quantitative and qualitative tools aim at better understanding possible futures and reducing uncertainty and ignorance. This reduced uncertainty needs to be embraced and managed in order to better shape the future and prepare various actors for it. In this context, combining methodological approaches creates opportunities to complement the weaknesses inherent to the use of one single approach with strengths of other approaches. In trying to push forward the field of FTA towards methodological integration, this paper has explored possible barriers hampering such development, and proposed a way forward. Overcoming the barriers identified calls for additional research efforts but also as importantly, for more practical experiment. Research has so far primarily concentrated on the development of innovative ICT tools and of ex-post interfaces, i e. mechanisms to bridge existing techniques of both types. New research efforts are required to devise methodological and conceptual frameworks, approaches and tools that intrinsically (ex-ante) integrate qualitative and quantitative thinking, as well as guidance for the 16 If forecasting is used to compare the impact of alternative policy options, validation could consist of comparing forecast results of the selected policy with the ex-post impacts of that policy. 394 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 identification of the features that may help the organisers of FTA projects in the selection of the most appropriate set of tools (characterising which tools fit best which contexts). On the other hand, the main barrier to integration appears to be the lack of communication and joint practice between the different FTA COMMUNITIES. As this is rooted in deep epistemological differences, it can hardly be reconciled on pure theoretical grounds. In the short term, efforts must be made to reduce the divide by promoting joint practice within actual FTA EXERCISES, establishing teams combining experts and practitioners from both communities from the start, and devoting more attention to communication processes. In the longer term, the conditions must be created (through education, information sharing, cultural dialogue) for a more integrated community of FTA EXPERTS practitioners, and users to emerge, where no divide needs to be brought down because no divide exists in the first place. Clearly this is an ambitious objective, which can only be reached through strong interactions within the FTA COMMUNITY and between it and the relevant stakeholders from education, policy and society at large. Acknowledgements The authors thank the anonymous reviewers and Ben Gardiner for commenting on this paper, and both the Scientific Committee and the participants of the 2011 Seville Conference on Future-oriented technology analysis for the fruitful discussions that helped shaping and refining it. References 1 F. Scapolo E. Cahill, New horizons and challenges for future-oriented technology analysis: proceedings of the EU US scientific seminar: new technology foresight, forecasting & assessment methods, in: JRC Technical Report, EUR 21473 EN, European commission, 2004, Available at: http://forera. jrc. ec. europa. eu/fta/Proceedings%20short%20version. pdf. Last accessed July 2012.2 P. Bazeley, Issues in mixing qualitative and quantitative approaches to research, in: R. Buber, J. Gadner (Eds. Applying Qualitative methods to Marketing Management Research, Palgrave Macmillan, Hampshire, United kingdom, 2004.3 J. Brannen, Mixed methods research: a discussion paper, NCRM Methods Review Papers, NCRM/005. Unpublished, 2005. Available at: http://eprints. ncrm. ac. uk/89/1/Methodsreviewpaperncrm-005. pdf, last accessed July 2012.4 J. E. M. Sale, L. H. Lohfeld, K. Brazil, Revisiting the quantitative qualitative debate: implications for mixed-methods research, Qual. Quant. 36 (1)( 2002) 43 53.5 C. Teddlie, A. Tashakkori, Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative approaches in the Social and Behavioral Sciences, Sage Publications, 2009.6 M. Wood, C. Welch, Are qualitative and quantitative useful terms for describing research? Methodol. Innov. Online 5 (1)( 2010) 56 71.7 M. Cooke, N. Buckley, Web 2. 0, social networks and the future of market research, Int. J. Mark. Res. 50 (2)( 2008) 267 292.8 S. Mathison, Why triangulate? Educ. Res. 17 (2)( 1988) 13 17.9 G. B. Rossman, B. L. Wilson, Numbers and words revisited: being shamelessly eclectic, Qual. Quant. 28 (3)( 1994) 315 327.10 V. J. Caracelli, J. C. Greene, Crafting mixed-method evaluation designs, in: J. C. Greene, V. J. Caracelli (Eds. Advances in Mixed-method Evaluation: The Challenges and Benefits of Integrating Diverse Paradigms, Jossey-Bass, San francisco, 1997.11 P. Goodwin, Why hindsight can damage foresight, Int. J. Appl. Forecast. 17 (2010) 5 7. 12 H. A. Linstone, Three eras of technology foresight, keynote speech, in: Fourth International Seville Conference on Future-oriented technology analysis, May 12 13 (2011) Seville, 2011, Available at: http://foresight. jrc. ec. europa. eu/fta 2011/documents/download/PRESENTATIONS/Keynotes/FTA%202011%20%205-9%20%20hl%20%20 %20%20%20ppt%20%20copy. ppt. Accessed July 2012.13 H. A. Linstone, Three eras of technology foresight, Technovation 31 (2011) 69 76.14 I. Tuomi, Foresight in an unpredictable world, Technol. Anal. Strateg. Manage. 24 (8)( 2012) 735 751.15 J. Calof, J. E. Smith, Critical success factors for government-led foresight, Sci. Public policy 37 (1)( 2010) 31 40.16 L. Georghiou, M. Keenan, Evaluation and impact of foresight, in: L. Georghiou, J. Cassingena Harper, M. Keenan, I. Miles, R. Popper (Eds.),The Handbook of Technology foresight: Concepts and Practice, Edward Elgar, Cheltenham, UK, 2008.17 A. Havas, D. Schartinger, M. Weber, The impact of foresight on innovation policy-making: recent experiences and future perspectives, Res. Eval. 19 (2)( 2010) 91 104.18 FTA Scientific Committee, Call for papers, in: The 4th International Seville Conference on Future-oriented technology analysis (FTA: FTA and Grand Societal Challenges Shaping and Driving Structural and Systemic transformations, 2011, Available at: http://foresight. jrc. ec. europa. eu/fta 2011/FTA2011 CALL FINAL. pdf. Accessed August 2012.19 H. Van Lente, Navigating foresight in a sea of expectations: lessons from the sociology of expectations, Technol. Anal. Strateg. Manage. 24 (8)( 2012) 769 782.20 D. Loveridge, O. Saritas, Ignorance and uncertainty: influences on FTA, Technol. Anal. Strateg. Manage. 24 (8)( 2012) 753 767.21 K. Haegeman, K. M. Weber, T. Könnölä, Preparing for grand challenges: the role of future-oriented technology analysis in anticipating and shaping structural and systemic changes, Technol. Anal. Strateg. Manage. 24 (8)( 2012) 729 734.22 A. Eerola, I. Miles, Methods and tools contributing to FTA: a knowledge-based perspective, Futures 43 (2011) 265 278.23 For-Learn, Online foresight guide, European foresight platform. Available at: www. foresight-platform. eu/community/foresightguide/2010accessed August 2012.24 J. E. Smith, O. Saritas, Science and technology foresight baker's dozen: a pocket primer of comparative and combined foresight methods, Foresight 13 (2)( 2011) 79 96.25 R. Popper, How are selected foresight methods, Foresight 10 (6)( 2008) 62 89.26 In: M. Keenan, R. Popper (Eds. Guide to Research infrastructures Foresight, European commission, Brussels, 2007.27 F. Scapolo, A l. Porter, New methodological developments in FTA, in: C. Cagnin, M. Keenan, R. Johnson, F. Scapolo, R. Barré (Eds. Future-oriented technology analysis: Strategic intelligence for an Innovative economy, Springer verlag, Berlin and Heidelberg, 2008.28 S w. Cunningham, T. E. van der Lei, Decision-making for new technology: a multi-actor, multi-objective method, Technol. Forecast. Soc. Change 76 (2009) 1037 1050.29 N. Agami, M. Saleh, H. El-Shishiny, A fuzzy logic based trend impact analysis method, Technol. Forecast. Soc. Change 77 (2010) 1051 1060.30 E. Kemp-Benedict, Converting qualitative assessments to quantitative assumptions: Bayes'rule and the pundits wager, Technol. Forecast. Soc. Change 77 (2010) 167 171.31 B. P. Bryant, R. J. Lempert, Thinking inside the box: a participatory, computer-assisted approach to scenario discovery, Technol. Forecast. Soc. Change 77 (2010) 34 49.395 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 32 D. Rossetti di Valdalbero, The Power of Science economic research and European decision-making: the case of energy and environment policies, Peter Lang, 2010. ISBN 978-90-5201-586-6 pb. 33 N. Shibata, Y. Kajikawa, Y. Takeda, I. Sakata, K. Matsushima, Detecting emerging research fronts in regeneration medicine by the citation network analysis of scientific publications, Technol. Forecast. Soc. Change 78 (2011) 274 282.34 H. M. Järvanpää, S. J. Mäkinen, M. Seppänen, Patent and publishing activity sequence over a technology's life cycle, Technol. Forecast. Soc. Change 78 (2)( 2011) 283 293.35 D. Thorleuchter, D. Van den Poel, A. Prinzie, A compared R&d-based and patent-based cross-impact analysis for identifying relationships between technologies, Technol. Forecast. Soc. Change 77 (2010) 1037 1050.36 R. Popper, Foresight methodology, in: L. Georghiou, J. C. Harper, M. Keenan, I. Miles, R. Popper (Eds.),The Handbook of Technology foresight, Concepts and Practicesedward Elgar, UK, 2008.37 C. S. Curran, J. Leker, Patent indicators for monitoring convergence examples from NFF and ICT, Technol. Forecast. Soc. Change 78 (2011) 256 273.38 P. Lee, H. Su, F. Wu, Quantitative mapping of patented technology the case of electrical conducting polymer composite, Technol. Forecast. Soc. Change 77 (2010) 466 478.39 K. Haegeman, C. Cagnin, T. Könnölä, D. Collins, Web 2. 0 foresight for innovation policy: a case of strategic agenda setting in European innovation, innovation: management, Policy Pract. 14 (3)( 2012) 449 469.40 F. K. Jin, W. R. Fah, N d. En, L. M. Wei, L . H. Wai, Developing a horizon scanning system for early warning, in: 12th International Command and Control Research and Technology Symposium, US Naval War College, Newport R i. USA, June 19 21,2007, 2007, Available at: http://www. dodccrp. org/events/12th iccrts/CD/html/papers/053. pdf. Accessed July 2012.41 A. Soojung-Kim Pang, Future 2. 0: rethinking the discipline, Foresight 12 (1)( 2010) 5 20.42 Y. Nugroho, O. Saritas, Incorporating network perspectives in foresight: a methodological proposal, Foresight 11 (6)( 2009) 21 41.43 B. Boyer, J. W. Cook, M. Steinberg, in: Studio: Recipes for Systemic change, Helsinki Design Lab Powered by Sitra, 2010, Available at: http://helsinkidesignlab. org/peoplepods/themes/hdl/downloads/In studio-Recipes for systemic change. pdf. Last accessed July 2012.44 J. Alcamo, D. van Vuuren, C. Ringler (Coordinating lead authors), Methodology for Developing the MA Scenarios, in: S. R. Carpenter, P. L. Pingali, E. M. Bennett and M. B. Zurek (Eds), Ecosystems and Human Well-being: Scenarios, Volume 2, Millennium Ecosystem Assessment, Washington: ISLAND PRESS, 2005.45 D. T. Campbell, A Study of Leadership Among Submarine Officers, Ohio State university Res. Found, Columbus, 1953.46 D. T. Campbell, D. W. Fiske, Convergent and discriminant validation by the multitrait multimethod matrix, Psychol. Bull. 56 (2)( 1959) 81 105.47 N. K. Denzin, The Research Act, 2d ed. Mcgraw-Hili, New york, 1978.48 T. D. Jick, Mixing qualitative and quantitative methods: triangulation in action, Adm. Sci. Q. 24 (4)( 1979) 602 611.49 E. Webb, D. T. Campbell, R. D. Schwartz, L. Sechrest, Unobtrusive Measures. Nonreactive Research in the Social sciences, Rand mcnally, Chicago, 1966.50 S. Sarantakos, Social research, Macmillan, Basingstoke, 1993.51 D. Silverman, Interpreting Qualitative data: Methods for Analysing Talk, Text and Interaction, Sage, London, 1993.52 K. R. Howe, Against the quantitative qualitative incompatibility thesis, or, Dogmas die hard, Educ. Res. 17 (1988) 10 16.53 K. R. Howe, Getting over the quantitative qualitative debate, Am. J. Educ. 100 (1992) 236 256.54 T. D. Jick, Process and Impacts of a Merger: Individual and Organizational Perspectives. Doctoral dissertation, New york state School of Industrial and Labor Relations. Cornell University, 1979.55 T. Lawson, Developments in economics as realist social theory, Rev. Soc. Econ. 405 (1996) 422.56 W. K. Olsen, Triangulation in social research: qualitative and quantitative methods can really be mixed, in: M. Holborn, Haralambos (Eds. Developments in Sociology, Causeway Press, 2004.57 A. Sayer, Method in Social science: A Realist Approach, Routledge, London, 1992. orig. 1984. 58 A. Baban, Reconceptualisation of the division between quantitative and qualitative researchmethods, cognition, brain, behavior, Interdiscip. J. XII (4)( 2008) 337 343.59 F. G. Castro, J. G. Kellison, S. J. Boyd, A. Kopak, A methodology for conducting integrative mixed methods research and data analyses, J. Mixed Methods Res. 4 (4)( 2010) 342 360.60 R. B. Johnson, A j. Onwuegbuzie, Mixed methods research: a research paradigm whose time has come, Educ. Res. 33 (7)( 2004) 14 26.61 A. Tashakkori, C. Teddlie, Mixed Methodology, Combining Qualitative and Quantitative approaches, Applied Social research Methods Series, 46, Sage Publications, 1998.62 A. Bryman, Barriers to integrating quantitative and qualitative research, J. Mixed Methods Res. 1 (1)( 2007) 8 22.63 J. Mahoney, G. Goertz , A tale of two cultures: contrasting qualitative and quantitative research, Pol. Anal. 14 (2006) 227 249.64 M. K. B. Lüdeke, Bridging Qualitative and Quantitative Methods in Foresight, Potsdam Institute for Climate Impact Research, 2006. Available at http://www. pik-potsdam. de/luedeke/lit/springer12 ch4. pdf. Last accessed July 2012.65 A j. Reiss, Stuff and nonsense about social surveys and participant observation, in: H. S. Becker, B. Geer, D. Riesman, R. S. Weiss (Eds. Institutions and the Person: Papers in Memory of Everett C. Hughes, Aldine, Chicago, 1968.66 L. Georghiou, J. C. Harper, Rising to the challenges Reflections on Future-oriented technology analysis, Technol. Forecast. Soc. Change 80 (3)( 2013) 467 470 (this issue. 67 T. Wehnert, W. Jörß, Evaluation paper: foresight and decision-making, in: Deliverable 3. 1 of EFONET Energy Foresight Network, 2009, Available at: http://www. efonet. org/index. php? option=com docman&task=doc download&gid=215&itemid=41. Accessed July 2012.68 R. Popper, M. Keenan, I. Miles, M. Butter, G. Sainz, Global Foresight outlook 2007: mapping foresight in Europe and the rest of the world, in: The EFMN Annual Mapping Report 2007, Report to the European commission, University of Manchester, TNO, Manchester/Delft, 2007.69 R. Popper, Mapping Foresight revealing how Europe and other world regions navigate into the future, in: EUR 24041 EN, European commission, Directorate-General for Research, Socioeconomic Sciences and Humanities, 2009, Available at: http://ec. europa. eu/research/social-sciences/pdf/efmnmapppingforesight en. pdf, last accessed July 2012.70 M. Rader, A l. Porter, Fitting future-oriented technology analysis methods to study types, in: C. Cagnin, M. Keenan, R. Johnson, F. Scapolo, R. Barré (Eds. Future-oriented technology analysis: Strategic intelligence for an Innovative economy, Springer verlag, Berlin and Heidelberg, 2008.71 R. Cameron, Mixed methods research: the five ps framework, Electron. J. Bus. Res. Methods 9 (2)( 2011) 96 108.72 O. Da Costa, P. Warnke, C. Cagnin, F. Scapolo, The impact of foresight on policy-making: insights from the FORLEARN mutual learning process, Technol. Anal. Strateg. Manage. 20 (3)( 2008) 369 387.73 B b. Hughes, P. D. Johnston, Sustainable futures: building policy options into a scenario for development in a global knowledge society, Futures 37 (2005) 813 831.74 R. Johnston, Developing the capacity to assess the impact of foresight, Foresight 14 (1 )( 2012) 56 68.75 H. White, Combining quantitative and qualitative approaches in poverty analysis, World Dev. 30 (3)( 2002) 511 522.76 C. Ansell, A. Gash , Collaborative governance in theory and practice, J. Public Adm. Res. Theory 18 (4)( 2008) 543 571. Karel Haegeman is a scientific officer at JRC-IPTS (European commission. He has masters in business economics and in marketing, and has worked previously in innovation policy, regional economic policy, project management and market research. He has been involved in a diversity of European-wide research projects (the most recent one on Visions for the European research area) and in organising several foresight trainings and International Seville Conferences on Future-oriented technology analysis. He has been publishing articles and reports on anticipatory and analytical research in support of European RTDI policy and on new methods and tools for FTA. Elisabetta Marinelli is a scientific officer at JRC-IPTS (Grant Holder 30. She is an economist by training and holds a Phd from the London School of economics and Political science in the field of Economic geography. She has researched and worked in innovation, education and local development policies. She has experience in quantitative and qualitative research methods and at the JRC-IPTS she is developing a specific interest in their combination and application in foresight exercises. She is involved currently in several FTA ACTIVITIES including the European foresight Platform and the ERA NET Rus project. 396 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397 Fabiana Scapolo holds a Phd on foresight methodologies and practices from the Manchester University (UK). ) She works at the European commission Directorate General Joint research Centre (JRC) in Brussels, where she is involved in the setting up of a corporate intelligence function called Anticipation at the JRC. This function is studying technological and societal trends and events, which may affect future European public policies by applying horizon scanning and foresight. She has worked in several foresight projects mainly in the advancement of the application of foresight as an instrument for policy-making formulation. Andrea Ricci is Vice president of ISIS Institute of Studies for the Integration of Systems, Rome. He received his engineering degree at Ecole Centrale (Paris). His key qualifications are Sustainability Policy analysis and impact assessment, and foresight studies. He coordinated many EU projects with a strong forward looking dimension, including PASHMINA (Paradigm Shifts Modelling and Innovative Approaches) and EFONET (Energy Foresight Network) and is rapporteur of the EC Working group Global Europe 2030 2050. He contributed to several ex-ante and ex-post Impact assessment studies of FP programmes and activities in the fields of Environment, Bio-based economy and Socioeconomic sciences and humanities. Professor Alexander Sokolov is Deputy Director of the Institute for Statistical Studies and Economics of Knowledge and Director of the International foresight Centre at the Higher School of economics (HSE) one of the most prominent national research universities in Russia. He authored over 120 publications in Russia and internationally devoted to Foresight and S&t and innovation policies; managed several national S&t foresight exercises in Russia, participated in dozens of international projects. Alexander Sokolov is a member of a number of high-level expert groups at the OECD and other international organisations. 397 K. Haegeman et al.//Technological forecasting & Social Change 80 (2013) 386 397

< Back - Next >

Overtext Web Module V3.0 Alpha
Copyright Semantic-Knowledge, 1994-2011