Science and Public policy February 2010 0302-3427/10/010051-11 US$12. 00 Beech tree Publishing 2010 51 Science and Public policy, 37 (1 february 2010, pages 51 61 DOI: 10.3152/030234210x484775; http://www. ingentaconnect. com/content/beech/spp User-driven innovation? Challenges of user involvement in future technology analysis Katrien De Moor, Katrien Berte, Lieven De Marez, Wout Joseph, Tom Deryckere and Luc Martens The shift from the traditional push towards more user-driven innovation strategies in the information and communications technologies domain has urged companies to place the user at the core of their innovation process in a more systematic way. In this paper we reflect on the implications of this new innovation context for traditional product development processes. Given these implications, two challenges are discussed that are crucial to true user-driven innovation, i e. the challenge of continuously involving the user and the need for tools to facilitate the integration of knowledge into the increasingly interdisciplinary development process. Drawing on our own experiences in the Interdisciplinary research On Mobile Applications and Services (ROMAS) project, which focused on future mobile applications in a living lab setting, we illustrate how the two challenges can be tackled. HE INDUSTRY THAT DEVELOPS informattio and communication technologies (ICT) has been challenged in various ways over the last few decades. Due to extensive convergeenc in the domains of communication, consumer electronics, computing and content on the one hand (Yovanof and Hazapis, 2008) and hyper-competition and increased market liberalization on the other, companies that aim to occupy or sustain a leading market position in the ICT industry have increasinngl been forced into accelerated product developmeen and into skipping important research stages. As a result there has been an explosion of nondisruuptiv innovations that are not always clearly different from other products on the market (De Marez and Verleye, 2004; Yovanof and Hazapis, 2008. In this changed context, many new products fail to‘cross the chasm'between the adoption segmeent that include innovators and some early adopteer on the one hand and the mass market on the other (Moore, 2002; De Marez and Verleye, 2004. Furthermore, traditional product development strategies are said to have crucial shortcomings since they are no longer able to guarantee the successful adoption and diffusion of new ICT. Although innovattio is considered traditionally to be a rather lineaar research -and price-driven process, this focus seems to have shifted over the years (Rosted, 2006), influenced by the altered role of the technology user as an important stakeholder. Confronted with almost unlimited choices, users'demands have become more sophisticated. Today's users increasingly seek out those products and experiences that fit their persoona and situational needs. Consequently, a clear insight into users'needs and experiences has becoom indispensable (De Marez and Verleye 2004; Veryzer and Borja de Mozota, 2005. T Katrien De Moor (corresponding author), Katrien Berte and Lieven De Marez are at MICT-IBBT, Department of Communicattio Sciences, University of Ghent, Korte Meer 7, 9000 Ghent, Belgium; Emails: Katrienr. Demoor@Ugent. be; Katrien. Berte@Ugent. be; Lieven. Demarez@Ugent. be; Tel:++32/9. 264.84.59; Wout Joseph, Tom Deryckere and Luc Martens are at Wica-IBBT, Department of Information technology, Universiit of Ghent, Gaston Crommenlaan 8 bus 201,9050 Gent, Belgiium Emails: Wout. Joseph@Intec. Ugent. be; Tom. Dertckere@Intec. Ugent. be; Luc. Martens@Intec. Ugent. be. This work was supported by the IBBT ROMAS (Research On Mobile Applications and Services) project, co-funded by the Interdisciplinary Institute for Broadband Technology (IBBT), and a consortium of companies (I-City, Microsoft and Concentra). This paper is revised a version of a paper presented at the Third International Seville Seminar on Future-oriented technology analysis: Impacts and Implications for Policy and Decision-making, held 16 17 october 2008 at Seville, Spain. User involvement in future technology analysis Science and Public policy February 2010 52 Indeed, although‘the consumer'has always been important, the rationale of involving the user has changed drastically. The new context has urged companies to put user needs at the core of their innovation strategies in a more systematic and structured way. Many authors have explored this shift from traditional push-to more pull-and userdriive approaches. Whereas the former are characterrize by technology-centred strategies and limited user involvement the latter acknowledge the crucial role of users in the innovation process (Rickards, 2003; Trott, 2003; Von Hippel, 1986; 2005). ) In this context we can also refer to policy action that suppoort user-driven innovation, such as the rise of living labs, which are user-driven innovation environmments and the launch of the European Network of Living Labs (ENOLL) in 2006. Although many other policy initiatives are embedded in this new innovaatio context, it remains difficult to create a meaningful synergy between users and technology in the field of ICT development. This paper therefore aims to discuss the integratiio challenges still to be found in this user-centred context. It is organized as follows: first, we expand on a number of theoretical perspectives on technoloog and society and the notion of user-driven innovattion We then explore the implications for traditional innovation and development processes. Given these implications, we then identify two imporrtan challenges for scholars and practitioners from a user-driven innovation perspective. As a complement to the literature, we draw on our own experiences in the ROMAS project, to illustrate how the two challenges can be tackled. Finally, we summarriz our findings and identify some opportunities for further research in this field. Theoretical perspectives Interplay between technological and societal forces The relationship between technology and society has already been studied from various perspectives. The idea of‘technological determinism'which consideer technology as the prime mover in transformation, and which propagates the industry's‘push'perspectiive has dominated the theoretical debate for several decades. It largely ascribes changes in society to technological advances, which are assumed thus to have important social consequences (Haddon et al.,2005). ) This theory of‘technological determinism'fits into the‘diffusion of innovations'framework (Rogers, 1995), which is dedicated to the adoption and diffusion of new technologies in society. Technollog adoption is assumed to follow a predictable path and is considered to be influenced by‘change agents'(e g. private firms, influential individuals etc.).In the theory of diffusionism, the first group of people who adopt the new technology (innovators and early adopters) are seen as catalysts for the successsiv waves of adoption. The final aim is to reach the rest of the market, to the point where the adoptiio rate has become so high that the innovation can be considered successful (this is referred to as the‘critical mass')(Rogers, 1995). A common criticism Katrien De Moor holds an MSC in communication sciences from Ghent University, Belgium. Currently, she works as a researcher at MICT (Research group for Media and Informattio and Communications technologies (ICT)( website<www. mict. be affiliated to the IBBT and Ghent University. Her research interests include: interdisciplinary research on quality of experience (Qoe) and quality of service (Qos) in mobile media environments, evaluation of user-and futureorieente innovation techniques in the ICT domain, and advannce in Living Lab methodologies. She is preparing a Phd thesis on the measurement of Qoe in a mobile media environment. Katrien Berte studied communication sciences at Ghent University, Belgium. After graduating in 2001, she worked for a commercial market research agency. She joined MICT in 2005. Her research interests and publications lie in the field of quantitative survey analysis, new media and advertissing She is currently working on a Phd thesis on advertisiin in a digital media environment based on the IBBT research project ADME (website<http://projects. ibbt. be/adme>).>Lieven De Marez obtained an MSC in communication sciennce (1999) and then an MSC in marketing (2000. He started his career as research assistant at the Department of Communication Sciences, Ghent University and has a Phd in communication sciences. The main contribution of this work involves the development of a‘segmentation forecastting tool for prior-to-launch prediction of adoption potentiial and the development of a blueprint for better introduction strategies for ICT innovations in today's volatile market environment. He is currently the research director of MICT-IBBT. He also teaches innovation research and new communication technologies at Ghent University. Wout Joseph holds a MSC in electrical engineering from Ghent University (2000. He started his career at the Departtmen of Information technology (INTEC), University of Ghent and received his Phd degree in 2005. His research focused on the measurement and modeling of electromagneeti fields around base stations for mobile communications related to the health effects of the exposure to electromagneeti radiation. Since October 2007, he has worked for IBBT-INTEC as a postdoctoral fellow (FWO-V Research Foundation, Flanders. His research interests include: electromaagneti field exposure assessment, propagation for wireless communication systems, antennas and calibration. In particular, he specializes in wireless performance analyssi and Qoe. Tom Deryckere received an MSC degree in electrical engineeerin (micro-and optoelectronics) from Ghent University in 2004. In the same year, he started as a research enginnee at IBBT (Ghent University) in the field of interactive media. His research interests include: interactive applicatioons personalization, recommendation systems and evaluation of Qoe and quality of service (Qos) in mobile environments. Luc Martens received an MSC in electrical engineering and a Phd from Ghent University in July 1986 and December 1990, respectively. From September 1986 to December 1990, he was a research assistant at INTEC, Ghent Universiity Since January 1991, he has been a member of the permanent staff of INTEC and is responsible for the reseaarc on experimental characterization of the physical layer of telecommunication systems and smart interactive services. His research group joined the IBBT in 2004. User involvement in future technology analysis Science and Public policy February 2010 53 of the diffusion theory has to do with its proinnovvatio bias and the assumed linearity of the innovaatio and adoption process. However, from the 1960s on this industry-push perspective was challenged by more human-centred paradigms that largely reject this notion of technologgica determinism and which point to the deviation of adoption curves from Rogers'theory. One of them is the social shaping of technology framework, which focuses on the daily use of technology and stresses the power of human actors and societal forces (Williams and Edge, 1996; Lievrouw, 2006. This social constructivist vision aims to make technollog development more user-and human-centred. Closely related to the social shaping perspective is the social construction of technology (SCOT) approoac (Bijker and Law, 1992), in which the concept of‘interpretative flexibility'is used to refer to the differences between individuals and social groups when it comes to giving meaning to technological development (Haddon et al.,2005; Lievrouw, 2006. In the SCOT perspective, it is assumed that negotiatiio between certain social groups influences the construction and emergence of new technologies (Bijker and Law, 1992; Haddon et al. 2005). ) Although both approaches emphasize the interactiio between technological and societal forces, they have been criticized for their rather linear social determiinism Other theories have a less linear view: e g. the actor-network theory (Latour, 1993), which states that technology and people are part of sociotechhnica networks, which influence the shaping, forms and uses of (new technologies. This and other approaches try to focus on technological developmeen from a mutual shaping or interactionism point of view (Lievrouw, 2006. They provide us with a theoretical basis for uniting the technology-centred with the user-or human-centred vision, since the successful adoption and diffusion of technology is ascribed to the continuous interaction between technoloogica and societal forces (Rickards, 2003; Trott, 2003; Boczkowski, 2004. User-driven innovation In this new context, the notion of user-led or userdriive innovation has assumed a prominent role. In current definitions,‘user-driven innovation'refers to the process of collecting a particular type of informattio about the user: it deals with insights both at an observable and a more latent level that are quite difficult to grasp (Rosted, 2006). As a result, userdriive innovation requires an interdisciplinary approach. Several approaches have been put forward for the collection of this type of knowledge. Hansson (2006) distinguishes two types of user-driven innovattio methods: voice of the customer methods and lead-user methods. Eric Von Hippel's work on‘lead users'(1986) can undoubtedly be regarded as pioneeerin in this respect. Furthermore, the traditional user-research tools (including methods such as foccu groups, surveys etc. have been supplemented by alternative analytical methods (e g. archetype reseaarch personas, scenarios, proxy technology assesssmen etc. from various disciplines (e g. design, foresight, fault tree analysis, anthropology etc. in order to support user-driven innovation. Whereas the so-called traditional methods usually focus on what people say and think, methods from other disciplines are used now to dig deeper into what people do or want (e g. ethnographic research, observaations user toolkits etc.),and feel or dream (e g. generative methods)( Sleeswijk Visser et al. 2007). ) Følstad (2008) situates the rise of living labs in this context of user-driven innovation. Living labs are innovation environments that provide full-scale test-bed possibilities for inventing, prototyping, interaactiv testing and marketing of (new) mobile technology applications (Schumacher and Niitamo, 2008; Følstad, 2008. They can be seen as humancenntri systemic innovation instruments, encouragiin the interaction between all stakeholders in the innovation process and facilitating the involvement of users as co-creators (Ballon et al. 2007). As discussse by Warnke and Heimeriks (2008: 74), systemmi innovation instruments are intended: to provide platforms for learning and experimeenting facilitate the management of interfacces foster new alignment of elements and stimulate demand articulation, strategy and visiio building. Contrary to other test platforms, living labs provide a more natural testing environment and strongly encouurag continuous and meaningful interaction betwwee developers/suppliers and users. However, this shift towards user-driven innovatiio also brings problems and challenges, such as the issue of the continuous involvement of users and the discrepancy between theory and practice in this respeect Although the user-driven innovation paradigm advocates an open perspective and stimulates the involvvemen of users from the early development stages onwards, this still contrasts sharply with the In current definitions, user-driven innovation refers to the process of collecting a particular type of information about the user: it deals with insights at an observable and at a more latent level that are quite difficult to grasp User involvement in future technology analysis Science and Public policy February 2010 54 narrow and technology-centric scope of many projects. Furthermore, there is a lack of integration of best practices and available methodologies, methood and tools into interdisciplinary user-driven innovattio research (e g. in the living lab setting)( Feurstein et al.,2008; Følstad, 2008. For example, in the early development stages it is often difficult to transcend users'limited powers of imagination: without having a fully developed ICT device at their disposal, users do not have a clear-cut idea of what they require, want or need. Limonnar and de Koning, 2005: 176) This challenge requires a consolidation of knowleddg and tools from various disciplines (e g. foresigght design, social sciences) and reinforces the role of policy-makers in the establishment of innovative experimentation and co-creation platforms. Another recurring integration challenge arises from the interdiscipplinar process itself and refers to the integratiio of knowledge. The following section discusses two of these challenges, which were underlying the objectives of the ROMAS project. Integration challenges Challenge 1: Continuous and adequate interaction with the user The first challenge concerns the need for the continnuou and adequate involvement of the user. Severra scholars have focused on the fact that there are still only a few companies that effectively involve the customer or user in the innovation process (Alam, 2002; Krisensson et al. 2004). ) Kristensson et al. 2004: 4 5) attribute this discrepancy betwwee theory and practice mainly to the lack of empirical evidence on the benefits of userinvollvemen and user-oriented strategies compared to traditional research and development. Although research has indicated that if new product developmmen fails, it usually goes wrong from the beginniin (Khurana and Rosenthal, 1998), user involvement is limited too often to just one or only the final stage (e g. usability testing, evaluation etc.)(Haddon et al. 2005). ) However, the benefits of involving users continuously have already been investigated: users can for example generate unique and valuable ideas for future products (Kristensson et al. 2004). ) User-driven innovation should thus go beyond merely asking users for feedback after the piloting phase or launch. Instead, users should be involved from the idea stage right up to the postlauunc evaluation stage. Furthermore, as userdriive innovation deals with those user insights (needs, expectations etc. that users cannot always easily articulate, development teams are forced now to explore new and interdisciplinary methodological tools. Challenge 2: Integration of knowledge: creating a synergy between users and technology The second challenge concerns the problem of integraatin the knowledge being gathered by multidiscipliinar teams, using either user-or technologycenntre methodologies. Although it is crucial that the user insights generated find their way into the development process, the adequate translation and transformation of user insights and requirements into more technical requirements (and vice versa), is still a challenge. The notion of translators is used also in this context (Veryzer and Borja De Mozota, 2005. In this respect it is relevant to mention the gap betwwee Qoe and Qos, two important concepts in the field of ICT development. Whereas the latter, which refers to‘general application service performance'(Soldani, 2006: 1), received a lot of attention in the past, it seems that Qoe has driven now taken over by the abovementioned shift from push to pull. Experiience are seen as a new source of value (Pine and Gilmore, 1999: 2) and the nature of users'experiience with new products can determine their success or failure (Crisler et al. 2004; Jain, 2004) In this changed context, Corrie et al. 2003: 2) emphasiiz the importance of users'expectations and experiences: Qoe is how the user feels about how an applicattio or service was delivered, relative to their expectations and requirements. De Marez and De Moor (2007) looked into Qoe at a conceptual level and identified five main dimensions and over 70 subdimensions. Given this diversity of factors influencing users'Qoe, its adequate measureemen and translation remains a challenge: insigght into users'experiences and expectations (e g. in a particular context or for a certain application) are shared often not in an interdisciplinary developmeen team. In this respect, the blueprint of a new interdiscciplinar approach for correlating Qoe to Qos parameters in a living lab environment is expanded upon in this paper. The next section illustrates how both challenges were tackled in the interdisciplinary ROMAS projeect The results are structured into three main reseaarc stages. Case study: The ROMAS project Project description and research setting ROMAS was funded by a consortium of industry partners (i e. Microsoft, Concentra and i-City) and the IBBT, founded by the Flemish Government in 2004 to stimulate innovation in the ICT domain. A user-oriented assessment of (future) wireless applicattion in cities was conducted. Six IBBT research groups collaborated on this project and i-City's User involvement in future technology analysis Science and Public policy February 2010 55 large-scale living lab was the main research location. Although it is now part of ilab (an IBBT research platform offering three complementary infrastructuure for elaborate testing in both controlled and liviin lab settings i-City was located in Hasselt (Belgium) at the time. Using technologies such as Wi-fi, Bluetooth, general pocket radio service (GPRS) and universal mobile telecommunications system (UMTS), it covered a wireless environment offering various applications to a large panel of test users via different platforms (e g. personal digital assisstan (PDA), laptop etc.).The main objective of ROMAS was to generate a set of cross-application research findings that can optimize the integrated development process for new digital products and services. In view of this, an interdiscciplinar framework was set up to pretest and co-develop new and innovative applications. This framework enables developers and companies to gain an insight into the main drivers and constraints in service innovation and into the conditions for meeting social and user requirements (Lievens and Pierson, 2006). Methodological framework The common methodological framework covered three main research stages in the innovationdevellopmen process (Lievens and Pierson, 2006. In the first stage (opportunity identification), possibbl (future) applications and trends in consumer behavviou were explored. Next, the socioeconomic feasibility of these possible services and applications was investigated by the i-City test panel. At the time of the research, the panel had 450 members. Althooug the test users were more than averagely intereeste in mobile technologies, the explorative nature of this project and the open access to the panel justified the choice of this research setting. In the second stage (concept development and evaluatioon) a selection of mobile applications was studied by interdisciplinary research teams in a horizontal layer investigating market-oriented, sociological, usability, legal, technological and social networking issues. Finally, the third stage (test market and pilotinng consisted of an evaluation of the results from the second stage on an individual application level and included an assessment of possible strategies for service innovation. In this paper, we only focus on user -and market-oriented research conducted by the Research group for Media and ICT and Wireless and Cable Research group, both affiliated to IBBT and Ghent University. In order to illustrate how the abovementioned integrratio challenges were tackled in ROMAS, we zoom in on distinct moments of user involvement during the three stages (see Figure 1), discussing each concrete methodological approach. In addition, we illustrate how a living lab setting can be complemented successfuull by other research methods. Results of stage 1: Opportunity identification The first stage, i e. the identification of mobile opportunnities started with a wide scan of mobile applicattion that are possibly of interest for a wireless city environment. The purpose of this scan was to generate relevant input and knowledge in order to identify current and future mobile applications that might not only make a significant difference to consummers everyday lives, but might also generate revenues for technology suppliers. One of the major challenges at this stage was not only to successfully involve users in this early part of the process, but also to overcome users'limited capacity to imagine future technological opportunities. First extensive desk research was conducted in order to list existing mobile applications and new concepts developed by the mobile industry. This invenntor was used as background information to familiiariz the researchers with the possibilities of mobile technologies. Secondly, in order to generate some new (and even wild) ideas for future mobile city services, users were involved in two focus groups. The first consisted of members of the i-City panel, all familiar with advanced mobile applications and their use in a city environment. The second group consisted of regular consumers, only familiar with the basic applications of traditional mobile phones. In this context a frequently recurring issue in user research is limited the ability of common users to break loose from the existing frame of reference and to imagine future needs and applications. Users oftte keep referring to familiar technologies such as multimedia messaging services, phone calls etc. and find it difficult to empathize with other users'lifestyyles e g. a 25-year-old reflects only on his daily Innovation-development process Prior-to-launch Post-launch R&d Opportunity identification Concept design Concept development and evaluation Innovation development and production Test market and piloting Launch Adoption diffusion User diffusion, evaluation Figure 1. Schematic overview of the three research phases User involvement in future technology analysis Science and Public policy February 2010 56 activities and finds it difficult to identify with the life of an elderly person. To eliminate these shortcomings, time-use frameworks were used in the focus groups. Eight categories of daily time-use were identified: social participation, household activities, study, work, transportation, leisure, health and sleeping and restinngrelaxing. To complement the traditional social science method of focus groups, three userarcheetype were developed to help the participants empathize with other lifestyles. Working with archetyype is an alternative way of conducting user reseaarc and was inspired by design practices. An example of one of the archetypes is Patricia (see Table 1). Patricia is 40 years old, a manager in a major international firm, mother of two children and therefore always trying to balance work and family life. For each archetype, we listed a series of daily activities within the time-use framework. Participants in the focus groups were asked to descrrib their daily activities at different times using questions like ‘what do you normally do on a work day between 7 and 9 am?''requiring simple answers like‘Take the kids to school, have breakfast, drive to work, take the bus to the university etc.''They then reflected on how mobile technologies could facilitaat these activities. The archetypes were used to refllec on the activities of people with other lifestyles. During the brainstorming session, which took place at the end of 2006, participants imagined they were in the year 2010 and were restricted therefore not by current legislation and technological limitations. 47‘wild ideas'were generated in these sessions, all original and very useful for subsequent stages of the research project. By combining the wild user ideas with the results from the desk research, a list of 80 mobile applications was created. The list was divided preliminnaril into eight categories based on the time-use research. Although the full list of 80 applicattion (Table 2) is too long to be discussed in detail in this paper, it served as input for the compilation of attractive and successful application clusters (stage 2). Given the results that were yielded in this first instaanc of user involvement, the next logical step was a feedback session with the supply-side, i e. the conteen and service providers who can also be considerre as professional users of the mobile applications and can therefore provide valuable input for technologgica developments. Potential service providers were contacted and sounded out for their interest in Table 2. Final list of 80 (future) mobile applications Finding people with same interests Note taking Reader Mobile payment Traffic jam alerts Mobile auction Practical and admin. Information for students Mapquest find me Prescriptions Indication of parking spaces and availability i-nanny Health monitoring Mobile information services Mobile flirt E-care Sports events on mobile Mobile chat Finding lost elderly person Tourist portal Mail/diary on mobile Video surveillance Keeping up hiking and cycling routes Mobile domotics Shared agenda Mobile news Mobile banking Business card exchange Find shops Parking ticket on mobile Meal help Movie choice E-ticket Mobile administration Download presentations or other info Shopper School diary and report Checking available places in cinema equick recipes Mobile academy Smart domotics E-meal Study mentor Smart machines on mobile Study choice guide Restaurant order and payment Medication prescriptions and schedules Mobile learning CV on mobile Public transport schedules Mobile terminal Accident reporting Carpooling system I-map Manual download Mobile dating Automated tolling Shop alert Smart mobile messenger Mobile blog Receipt download‘Independent living'support Mobile feed reader Monitoring organization aid Free mobile surfing Restaurant finder Identity and medical info on mobile Location-based advertising Photo service Heartbeat information Making appetizers Scanning information Dentist appointment Mobile video calling Webcam Blind aid Spare time suggestions Museum tour Cot death alarm Mobile search Event information Table 1. Archetype Patricia and some of her daily activities Time use Specific activity Housekeeping activities Cleaning Contact the cleaning lady Food Buy groceries Children's education Help the children with their homework Work Full-time job Keep up with email Contact employer when child is ill Book a ticket for a conference Meet a business associate in a restaurant Transportation Journey from home to work Avoid traffic jams Buy petrol Take the children to school Travel abroad Book airline tickets Leisure travel Go on holiday User involvement in future technology analysis Science and Public policy February 2010 57 the use of mobile applications to support their existiin products and services. By combining all input sources (desk research, user research and consultatiio of industry partners), three types of applications were identified: User-generated applications that were not being developed by the industry at the time of the reseaarch Applications that were mentioned by the users and that were already being implemented by the industry. Applications that were mentioned not by the users in the study, but that already were already commerciialize or being developed by the industry (push-driven). Results of stage 2: Concept evaluation Next, all the applications considered in the first phase were transformed into workable paper conceept and presented to a large audience in order to evaluate their adoption potential and identify appropriiat market segments. For this concept evaluation phase, we conducted a major survey among the i-City panel (n=420. The advantage of such an evaluation is the panel's familiarity with mobile city concepts and experience with actual working applicatiions In total, 312 panel members completed the survey. First, we examined the 80 applications and/or ideas and tried to group the long list into some clearly distinguishable application clusters. The criteeri for this clustering are the correlations and similariitie in interest patterns for certain subsets of applications. The difference in the interest shown in each of these application clusters can be considered as a first exploration of their potential. Secondly, the clusters were ranked to identify the most promising application (s)/ cluster (s). A factor analysis (using the principal component analysis technique) of the interest shown by the 312 respondents in the 80 mobile city application (s) showed that this interest can be summarized in 21 factors, still explaining 67.5%of the total variance (R2=0. 674). The internal consistency and reliability of each factor was assessed using Cronbach's alpha (alpha values exceeding 0. 65 are considered to be internnall consistent. Using this approach, 13 clusters were discovered (Table 3) . Since each cluster represents a set of applications with strong correlations in the interest (or disinteresst shown by the 312 respondents, they can also be considered as a potential added value domain for mobile city applications for a certain part of the market. 16 applications could not be clustered and were analysed therefore separately. Clusters and single applications were ranked on the basis of the respondennts interest level (Table 4). The overall average interest ranking for all the clusters showed that the most important innovations in these mobile applicatiion are not the most high-tech ones but rather those applications that enable people to save time and that contribute to a better quality of life. Despite the popularity of the virtual social contacts on the web, the mobile social contacts and friends cluster certainly does not appeal to the majority of the samppl population (2. 94/5). ) Despite the high ranking of news in the most-wanted content rankings, the mobiil news cluster only received an average interest of 3. 11/5. There does not seem to be great enthusiasm for sports on mobile (2. 74/5) either. A possible explannatio for this may be found in the somewhat abstrrac description of the application ideas which is typical for this stage of the process. Further and deeper analysis is certainly necessary in order to reach definite conclusions about the appeea of each of these applications. The correlation of interests in the different kinds of application was used also to cluster participants. Although this clusterrin is discussed not in detail in this paper, the analysis yielded valuable insights into the appeal of certain types of application to certain user clusters, by means of profound analysis of interest, perceived added value and willingness-to-pay for some applicatiions Results of stage 3: Test market For the third user involvement session, we take a look at the test market phase. One application, i e. mobile news, was selected from the list by the industtr partners for further development despite its limitte appeal to the panel members (Table 4). As the industry partners aimed to test the application and assess the adoption potential of the mobile news application on the basis of a working prototype, the mobile news application was developed by Concenntra The i-City panel was able to test it and the adoption potential of the application was assessed by means of a large-scale survey (Berte et al.,2008). ) Since the results from earlier user research were Table 3. 13 clusters and corresponding Cronbach's alpha values Application cluster Cronbach's alpha Food and shopping help 0. 871 Tourist information 0. 775 Mobile social contact and friends 0. 789 Doing‘usual, daily tasks'more efficiently by mobile 0. 812 More effective healthcare 0. 812 Mobile high-tech 0. 790 Mobile help for study and work 0. 764 Doing unusual tasks more effectively by mobile 0. 776 Payment and Money affairs 0. 763 Help with serious health issues 0. 721 Multimedia 0. 654 Administration 0. 760 Mobile news and information 0. 679 User involvement in future technology analysis Science and Public policy February 2010 58 disregarded, this choice illustrates that decisions are made sometimes at the expense of the user-centred rationale. At this stage in particular, other factors were considered to be of greater importance to the decision to be taken by the project's industrial partneers These factors included some of the following: the substantial involvement of the inhabitants of Hasselt in their local community, the role of existing local news initiatives, the presence of a community of city reporters and the agenda of one of the industriia partners, which was a local content provider (Concentra) with a proper local TV news channel (TV Limburg) and which expressed the need for a cross-media approach. During the same stage, the Qoe of one particular application (i e. Wapedia: a Wikipedia application for mobile access) was investigated in a controlled reseaarc setting. This study should be seen as a methodoloogica sideline in the ROMAS project, focusing on the evaluation of Qoe in a mobile living lab settiing In this context, we developed a five-step interdiscipplinar approach for linking Qoe to Qos parameters in living lab environments. This approach draws on hard technical parameters as well as more subjective (social, contextual etc. elements and their translation. As a result, it not only considers the questiio of what is happening (e g. on the network), but also the matter of why certain things are happening (e g. why does the user feel frustrated?).It was tested in a pilot study of the Wapedia application. Ten test users, all recruited by a specialized recruiting agency, were involved. Instead of elaborating on the results (see Deryckere et al. 2008), this paper limits itself to a discussion of the research process and the way the abovementioned challenges were tackled. We now briefly turn to the five stages. 1. Pre-usage user research to detect relevant Qoe elements and user expectations. This stage incluude a semi-qualitative group session per two participants, who were asked to reflect on elemeent and factors influencing their experience with and expectations of mobile phone use. Methood used here included: free listing, questionnaire, brainstorming, Qoe elements list, prioritizing exerccis (card sorting) and conjoint analysis. The latter is used to determine which product features or attributes are considered to be most important when a set of them are offered. It allows us to analyse the preferences of the respondents. In our study, a set of six Qoe elements was offered, resulltin in a total of 15 combinations. Having calcullate the mean scores of these elements, the following top five ranking was obtained: 1. availability of network (connection at any time, anywhere; 2. user-friendliness; 3. interface; Table 4. Ranking of application clusters and separate applications based on respondents'interest level Application (cluster) Average interest (1: not interesting at all 5: very interesting) Application (cluster) Average interest (1: not interesting at all 5: very interesting) Application (cluster) Average interest (1: not interesting at all 5: very interesting) Very appealing Moderately appealing Not appealing Indication of parking spaces and availability 4. 23/5 Mobile search 3. 78/5 Food and shopping help (Foodshop cluster) 3. 23/5 Practical and administrative information for students 4. 20/5 Doing‘usual, daily tasks'more effectively by mobile (Effective I cluster) 3. 73/5 Mobile news and information (Mobnews cluster) 3. 11/5 Public transport schedules 4. 11/5 Checking available places in cinema 3. 72/5 Spare time suggestions 3. 10/5 Payments and financial affairs (Money cluster) 4. 01/5 More effective health care (Health I cluster) 3. 68/5 Mobile social contacts and friends (Social cluster) 2. 94/5 Traffic jam alerts 4. 01/5 Doing‘unusual tasks'more effectively by mobile Effective II cluster) 3. 68/5 Carpooling system 2. 93/5 Help with serious health issues (Health II cluster) 3. 99/5 Download presentations or other information 3. 65/5 Location-based advertising 2. 78/5 Independent living support 3. 93/5 Administration (Administration cluster) 3. 63/5 Sports events on mobile 2. 74/5 Free mobile surfing 3. 92/5 Multimedia (Multimedia cluster) 3. 57/5 Find shops 3. 92/5 Movie choice 3, 54/5 Tourist information (Tourist cluster) 3. 87/5 Mobile help for studies (and Work study cluster) 3, 43/5 User involvement in future technology analysis Science and Public policy February 2010 59 4. battery lifetime plus security; and 5. response time. 2. Pre-usage translation workshops. The aim of these workshops is to find an optimal match betwwee user-indicated Qoe elements and measurabbl Qos parameters. The intention of this stage is to translate insights from the user research into workable requirements. In the pilot study, a photo-download application was for example devellope to simulate different download times (ranging from 0 to 5 second scenarios. This appliccatio was shown to the test users, who were asked to indicate those scenarios that were accepttabl to them (for a good experience) in a mobiil context. 3. Monitoring of Qos parameters during use. In this stage, the respondents tested the selected applicatiion Several usage scenarios had to be carried out, consisting of a number of tasks to perform with a PDA. These tests were performed in an indooo environment at four different locations with a different signal strength at each location. The four locations were at different distances from the access point, corresponding with different signal strengths (dbm). By using several scenarios, the influence of repeated tests was minimized. The test users were not aware that the signal strength was manipulated. 4. Post-usage questions on device (e g. PDA. Immediiatel after the completion of each scenario, the test users were asked to fill in a short experiennc questionnaire of six questions (five-point scales), displayed on the PDA. The monitored signal strength and responses were saved on the PDA and automatically transmitted to the server. This resulted in 60 samples per location (total of 240 samples. 5. Post-usage comparison of expectations versus experience (based on information gathered in step 3 and further user research) in order to identify and explain differences/matches between them. In this stage, a similar methodological approach was taken as in stage 1. In this case-study, it was only signal strength that was perceived related to experiience with the aim of showing that there might be a relation between Qoe and Qos. For example, we selected user 10 (male, 33 years old) to expllai the results for an individual user. Figure 2 shows the rating of the answers given by user 10 to several questions (Q1 Q2, Q5, and Q6) as a function of the median signal strength at the differren indoor locations (location 1 corresponds to a median P=-43 dbm and location 4 correspoond to-83 dbm for this user). User 10 shows great satisfaction up to-79 dbm, with ratings of 5 for expectations, reuse and general experience. At-79 dbm a slight reduction in speed is noticed by this user due to the much lower signal strength: more time is needed to load such things as pictuure on the PDA, which causes the application to slow down. The ratings for speed and general experiienc drop significantly at-83 dbm. Expectatiion and reuse remain relatively high for user 10, and despite the bad experience at-83 dbm, user 10 would still reuse this application. In this last section our aim is to illustrate how Qoe might be measured by an interdisciplinary team and how insights from user research might be translated adequaatel into technical requirements. Future research will include the testing of this multimettho approach with a large number of users and several usage contexts and parameters in a living lab setting. Figure 2 Ratings for user 10 on questions Q1 Q2, Q5 and Q6 User involvement in future technology analysis Science and Public policy February 2010 60 Conclusion In this paper, we have focused on the shift from traditiiona technology push to more user-oriented and user -led approaches in the communications industry. Drawing on a number of theoretical frameworks that have studied the relationship between technology and users/society in greater depth, we reflected on the implications of this new innovation context for traditional development processes. It was mentioned that this predominant focus on the user led to an expannsio of the traditional range of user research instruument with methods and tools from other fields. However it was argued that, notwithstanding this ongoing broadening and despite the emphasis on such interdisciplinary approaches, it still remains difficult to create a meaningful synergy between useer and technology. Given the implications of the notion of userdriive innovation and the traditional tension betwwee user-and technology-centred strategies, two crucial challenges were identified: the need for continnuou interaction with the user and the need for mechanisms to integrate the knowledge that is gatherre in the increasingly multidisciplinary developmeen process. Empirical findings from the interdisciplinary ROMAS project on future mobile applications were presented in this paper in order to illustrate how both challenges can be tackled. Drawiin on results selected from three distinct points of user involvement in the process of developing new products (i e. opportunity identification, concept evaluation and test market), it was illustrated how research in a living lab setting can be complemented by other research methods in order to fuel the userdriive approach. However it was shown also that in some cases there is still a discrepancy between theory and practiice Although in theory many projects start from a user-and pull-driven perspective, the mantra that‘innovation should start with the user and end with the user'is pursued not always. At the policy level, considerable effort has already been put into the creation of a new innovation system. In this respect, the support for the ENOLL, which already includes around 130 members from various member states, illustrates that the creation, integration and harmonizaatio of these systematic innovation instruments is high on the agenda. But there is always room for improvement. From a methodological point of view there is a lack of robust tools to enable context and co-creation research in living labs. Furthermore, even in living lab research the focus is still primarily on a certain technology or new application (e g. mobile TV), rather than on the way users interact with different access networks in their natural environmment In this respect, the establishment of real user-driven living labs might provide a more accuraat insight into users'current and future needs. Puttiin user panels together on the basis of the devices and services people have adopted already and domesticated could be one way of doing this. In such a scenario new applications that are tested in the natural environment would therefore only be an additional layer on top of the‘domesticated'netwoork and devices. This could be the next step towaard a real user-driven innovation system. References Alam, I 2002. An exploratory investigation of user involvement in new service development. Journal of the Academy of Marketiin Sciences, 30 (3), 250 261. Ballon, P, J Pierson and S Delaere 2007. Fostering innovation in networked communications: test and experimentation platfoorm for broadband systems. In Designing for Networked Communications: Strategies and Development, S b Heilesen and S Siggaard Jensen (eds. pp 137 166, London: Hershey (PA. Berte, K, K De Moor, L De Marez, W Joseph, T Deryckere and L Martens 2008. User-driven innovations? Reassessing the value of bottom-up approaches within an interdisciplinary mobiil media context. Paper presented at European Communicatiion Policy Research Conference (Eurocpr), held 31 march 1 april 2008, Seville, Spain. Bijker, W E and J Law (eds. 1992. Shaping technology/Building society: Studies in Socio-Technical change. Cambridge, MA: MIT Press. Boczowski, P J 2004. The mutual shaping of technology and sociiet in videotex newspapers: beyond the diffusion and social shaping perspectives. The Information society, 20 (4), 255 267. Corrie, B H, T Wong, S Zimmerman, A s Marsh, J Patrick et al. 2003. Towards quality of experience in advanced collaborative environmments Paper presented at the Third Annual Workshop on Advanced Collaborative Environments, held 22 june 2003, Seattlle WA, USA. Crisler, K, T Turner, A Aftelak, M Visciola, A Steinhage et al. 2004. Considering the user in the wireless world. IEEE Communiccation Magazine, 42 (9), 56 62. De Marez, L and G Verleye 2004. Innovation diffusion: The need for more accurate consumer insight. Illustration of the PSAP scale as a segmentation instrument. Journal of Targeting, Measurement and Analysis for Marketing, 13 (1), 32 49. De Marez, L and K De Moor 2007. The challenge of user-and Qoe-centric research and product development in today's ICT environment. Observatorio (OBS*)online journal, 1 (3), 1 22. Deryckere T, W Joseph, L Martens, L De Marez, K De Moor et al. 2008. A software tool to relate technical performance to user experience in a mobile context. Proceedings of the 2008 Internatiiona Symposium on a World of Wireless Mobile and Multimmedi Networks, held 23 26 june 2008, Newport Beach, CA, USA. Feurstein, K, A Hesmer, K A Hribernik, K-D Thoben, J Schumacher 2008. Living Labs: A New development Strategy. In European Living Labs: A new approach for human centric regional innovation, J Schumacher and V-p Niitamo (eds. pp 1 14. Berlin: Wissenschaftlicher Verlag. Følstad, A 2008. Towards a living lab for the development of online community services. The Electronic Journal for Virtual Organizations and Networks, 10,47 58. Haddon, L, E Mante, B Sapio, K-H Kommonen, L Fortunati and A Kant (eds. 2005. Everyday Innovators: Researching the role of users in shaping ICTS. Dordrecht, The netherlands: Springer verlag. Hansson, E 2006. Understanding User-Driven innovation. Nordeen Unpublished report. Available at<http://www. norden. org/pub/velfaerd/naering/sk/TN2006522. pdf>,last accessed Februuar 2008. Jain, R 2004. Quality of Experience. IEEE Multimedia, 11 (1), 95 96. Khurana, A and S R Rosenthal 1998. Toward holistic‘front end'in new product development. Journal of Product innovation Managemment 15 (1), 57 74. Kristensson, P, A Gustafsson and T Archer 2004. Harnessing the creative potential among users. Journal of Product innovation Management, 21 (1), 4 14. User involvement in future technology analysis Science and Public policy February 2010 61 Latour B 1993. We Have Never Been Modern. New york: Harvester Wheatsheaf. Lievens, B and J Pierson 2006. Project Outline: GBO project 2006 ROMAS: Research on Mobile Applications and Services (interrna document. Lievrouw, L 2006. New media design and development: diffusions of innovations v social shaping of technology. In The Handbooo of New Media, L Lievrouw and S Livingstone (eds. pp 246 265. London: Sage. Limonard, S and N de Koning 2005. Dealing with dilemmas in pre-competitive ICT development projects: the construction of‘the social'in designing new technologies. In Everyday Innovatoors Researching the Role of Users in Shaping I's, L Haddon, E Mante, B Sapio, K-H Kommonen, L Fortunati and A Kant (eds. pp 168 183. Dordrecht, The netherlands: Springer verlag. Moore, G A 2002. Crossing the Chasm: Marketing and Selling Technology Products to Mainstream Customers. Oxford, UK: Capstone. Pine, J B and J H Gilmore 1999. The Experience Economy: Work is Theatre and Every Business a Stage. Boston, MA: Harvard university Press. Rickards, T 2003. The future of innovation research. In The Internatiiona Handbook on Innovation, L V Shavinina (ed.),pp 1094 1100. Oxford, UK: Pergamon/Elsevier. Rogers, E M 1995. Diffusion of innovations (4th edn..New york: The Free Press. Rosted, J 2006. User-driven innovation: an introduction. Presentattio at the Northern Dimension Learning Forum on User-Driven innovation. Available at<http://www. foranet. dk/upload/user-driven innovation 22 03 2006. pdf, >last accessed March 2007. Schumacher, J and V-p Niitamo (eds. 2008. European Living Labs. A New approach for Human Centric Regional Innovatiion WVB, Berlin. Sleeswijk Visser, F, R van der Lugt and P J Stappers 2007. Shariin user experiences in the product innovation process: Particippator design needs participatory communication. Journal of Creativity and Innovation Management, 16 (1), 35 45. Soldani, D 2006. Means and methods for collecting and analysing Qoe measurements in wireless networks. Proceedings of the International Symposium on a World of Wireless, Mobile and Multimedia Networks (WOWMOM. 2006.61), held 26 29 june 2006, Niagara-falls, Buffalo-NY, USA. Trott, P 2003. Innovation and Market research. In The Internatioona Handbook on Innovation, L V Shavinina (ed.),pp 835 844. Oxford, UK: Pergamon/Elsevier. Veryzer, R W and B Borja de Mozota 2005. The impact of userorieente design on new product development: an examination of fundamental relationships. The Journal of Product Innovatiio Management, 22 (2), 128 143. Von Hippel, E 1986. Lead users: a source of novel product conceppts Management Science, 32 (7), 791 805. Von Hippel, E 2005. Democratizing Innovation. Cambridge, MA: MIT Press. Williams, R and D Edge 1996. The social shaping of Technology research Policy, 25 (6), 865 899. Warnke, P and G Heimeriks 2008. In Future-oriented technology analysis: Strategic intelligence for an Innovative economy, C Cagnin, M Keenan, R Johnston, F Scapolo and R Barré (eds. pp 71 87. Berlin: Springer verlag. Yovanof, G S and G N Hazapis 2008. Disruptive technologies, services, or business models? Wireless Personal Communicatioons 45 (4), 569 583

< Back - Next >

Overtext Web Module V3.0 Alpha
Copyright Semantic-Knowledge, 1994-2011