Synopsis:
Ict: Data: (*)data_mining: Sensitivity analysis:
42495745.pdf
115 Step 7. Uncertainty and sensitivity analysis...117 7. 1 General framework...118 7. 2 Uncertainty analysis (UA...
118 7. 3 Sensitivity analysis using variance-based techniques...121 7. 3. 1 Analysis 1...124 7. 3. 2 Analysis 2...129 Step 8. Back to the details...
and Sensitivity analysis should be undertaken to assess the robustness of the composite indicator in terms of, e g.,, the mechanism for including
and sensitivity analysis Should be undertaken to assess the robustness of the composite indicator in terms of e g.,, the mechanism for including
To conduct sensitivity analysis of the inference (assumptions) and determine what sources of uncertainty are more influential in the scores
To correlate the composite indicator with other relevant measures, taking into consideration the results of sensitivity analysis.
To the extent that data permit, the accuracy of proxy measures should be checked through correlation and sensitivity analysis.
and sensitivity Sensitivity analysis can be used to assess the robustness of composite indicators Several judgements have to be made
A combination of uncertainty and sensitivity analysis can help gauge the robustness of the composite indicator
Sensitivity analysis assesses the contribution of the individual source of uncertainty to the output variance. While uncertainty analysis is used more often than sensitivity analysis
and is treated almost always separately, the iterative use of uncertainty and sensitivity analysis during the development of a composite indicator could improve its structure (Saisana et al.,
2005a; Tarantola et al. 2000; Gall, 2007. Ideally, all potential sources of uncertainty should be addressed: selection of individual indicators, data quality, normalisation, weighting, aggregation method, etc.
The sensitivity analysis results are shown generally in terms of the sensitivity measure for each input source of uncertainty.
The results of a sensitivity analysis are shown often also as scatter plots with the values of the composite indicator for a country on the vertical axis
Conducted sensitivity analysis of the inference, e g. to show what sources of uncertainty are more influential in determining the relative ranking of two entities.
Tested the links with variations of the composite indicator as determined through sensitivity analysis. Developed data-driven narratives on the results Documented
as well as the need to test the robustness of the composite indicator using uncertainty and sensitivity analysis.
The role of the variability in the weights and their influence on the value of the composite are discussed in the section on sensitivity analysis. 100 HANDBOOK ON CONSTRUCTING COMPOSITE INDICATORS:
AND SENSITIVITY ANALYSIS Sensitivity analysis is considered a necessary requirement in econometric practice (Kennedy, 2003) and has been defined as the modeller's equivalent of orthopaedists'X-rays.
This is what sensitivity analysis does: it performs the‘X-rays'of the model by studying the relationship between information flowing in and out of the model.
More formally, sensitivity analysis is the study of how the variation in the output can be apportioned, qualitatively or quantitatively, to different sources of variation in the assumptions,
Sensitivity analysis is thus closely related to uncertainty analysis, which aims to quantify the overall uncertainty in country rankings as a result of the uncertainties in the model input.
A combination of uncertainty and sensitivity analysis can help to gauge the robustness of the composite indicator ranking
Below is described how to apply uncertainty and sensitivity analysis to composite indicators. Our synergistic use of uncertainty and sensitivity analysis has recently been applied for the robustness assessment of composite indicators (Saisana et al.
2005a; Saltelli et al. 2008) and has proven to be useful in dissipating some of the controversy surrounding composite indicators such as the Environmental Sustainability Index (Saisana et al.
and sensitivity analysis discussed below in relation to the TAI case study is only illustrative. In practice the setup of the analysis will depend upon which sources of uncertainty and
()c Rank CI is an output of the uncertainty/sensitivity analysis. The average shift in country rankings is explored also.
The investigation of()c Rank CI and S R is the scope of the uncertainty and sensitivity analysis. 41 7. 1. General framework The analysis is conducted as a single Monte carlo experiment,
A scatter plot based sensitivity analysis would be used to track which indicator affects the output the most
such as the variance and higher order moments, can be estimated with an arbitrary level of precision related to the size of the simulation N. 7. 3. Sensitivity analysis using variance-based techniques A necessary step
when designing a sensitivity analysis is to identify the output variables of interest. Ideally these should be relevant to the issue addressed by the model.
2008), with nonlinear models, robust, model-free techniques should be used for sensitivity analysis. Sensitivity analysis using variance-based techniques are model-free
and display additional properties convenient in the present analysis, such as the following: They allow an exploration of the whole range of variation of the input factors, instead of just sampling factors over a limited number of values, e g. in fractional factorial design (Box et al.
They allow for a sensitivity analysis whereby uncertain input factors are treated in groups instead of individually; They can be justified in terms of rigorous settings for sensitivity analysis.
To compute a variance-based sensitivity measure for a given input factor i X, start from the fractional contribution to the model output variance,
The I s, Ti S, in the case of nonindependent input factors, could also be interpreted as settings for sensitivity analysis. 124 HANDBOOK ON CONSTRUCTING COMPOSITE INDICATORS:
Figure 19 shows the sensitivity analysis based on the first-order indices. The total variance in each country's rank is presented
The sensitivity analysis results for the average shift in rank output variable (equation (38)) is shown in Table 40.
and the testing of the robustness of the composite using uncertainty and sensitivity analysis. The present work is perhaps timely,
In Sensitivity analysis (eds, Saltelli A.,Chan K.,Scott M.)167-197. New york: John Wiley & Sons.
Homma T. and Saltelli A. 1996), Importance measures in global sensitivity analysis of model output, Reliability Engineering and System Safety, 52 (1), 1-17.
Saisana M.,Nardo M. and Saltelli A. 2005b), Uncertainty and Sensitivity analysis of the 2005 Environmental Sustainability Index, in Esty D.,Levy M.,Srebotnjak T. and de Sherbinin
. and Tarantola S. 2008), Global Sensitivity analysis. The Primer, John Wiley & Sons. Saltelli A. 2007) Composite indicators between analysis and advocacy, Social Indicators Research, 81:65-77.
Saltelli A.,Tarantola S.,Campolongo F. and Ratto M. 2004), Sensitivity analysis in practice, a guide to assessing scientific models, New york:
Software for sensitivity analysis is available at http://www. jrc. ec. europa. eu/uasa/prj-sa-soft. asp.
11-30 Sobol'I. M. 1993), Sensitivity analysis for nonlinear mathematical models, Mathematical Modelling & Computational Experiment 1: 407-414.
Tarantola S.,Jesinghaus J. and Puolamaa M. 2000), Global sensitivity analysis: a quality assurance tool in environmental policy modelling.
Sensitivity analysis, pp. 385-397. New york: John Wiley & Sons. Tarantola S.,Saisana M.,Saltelli A.,Schmiedel F. and Leapman N. 2002), Statistical techniques and participatory approaches for the composition of the European Internal Market Index 1992
EUR 21682 EN.pdf
81 7. UNCERTAINTY AND SENSITIVITY ANALYSIS 85 7. 1 Set up of the analysis 87 7. 1. 1 Output variables of interest 87 7. 1. 2
7. 1. 5 Normalisation 88 7. 1. 6 Uncertainty analysis 89 7. 1. 7 Sensitivity analysis using variance-based techniques 91 7
and aggregating indicators into a composite and test the robustness of the composite using uncertainty and sensitivity analysis.
or imprecise assessment and use uncertainty and sensitivity analysis to gain useful insights during the process of composite indicators building, including a contribution to the indicators'quality definition and an appraisal of the reliability of countries'ranking.
and sensitivity analysis to increase transparency and make policy inference more defensible. Section 8 shows how different visualization strategies of the same composite indicator can convey different policy messages.
Uncertainty analysis and sensitivity analysis is a powerful combination of techniques to gain useful insights during the process of composite indicators building,
A combination of uncertainty and sensitivity analysis can help to gauge the robustness of the composite indicator
Sensitivity analysis (SA) studies how much each individual source of uncertainty contributes to the output variance. In the field of building composite indicators, UA is adopted more often than SA (Jamison and Sandbu, 2001;
The iterative use of uncertainty and sensitivity analysis during the development of a composite indicator can contribute to its well-structuring
The role of the variability in the weights and their influence in the value of the composite will be the object of the section on sensitivity analysis (section 7). Table 6. 6. Weights for the sub-indicators obtained using 4 different methods:
and not importance coefficients 7. Uncertainty and sensitivity analysis The reader will recall from the introduction that composite indicators may send misleading,
A combination of uncertainty and sensitivity analysis can help to gauge the robustness of the composite indicator,
Uncertainty Analysis (UA) and Sensitivity analysis (SA. UA focuses on how uncertainty in the input factors propagates through the structure of the composite indicator
()c Rank CI will be an output of interest studied in our uncertainty sensitivity analysis. Additionally, the average shift in countries'rank will be explored.
and sensitivity analysis (both in the first and second TAI analysis), targeting the questions raised in the introduction on the quality of the composite indicator.
We anticipate here that a scatter-plot based sensitivity analysis will allow us to track which indicator when excluded affects the output the most.
such as the variance and higher order moments, can be estimated with an arbitrary level of precision that is related to the size of the simulation N. 7. 1. 7 Sensitivity analysis using variance-based techniques A necessary step
when designing a sensitivity analysis is to identify the output variables of interest. Ideally these should be relevant to the issue tackled by the model,
In the following, we shall apply sensitivity analysis to output variables()c Rank CI, and S R, for their bearing on the quality assessment of our composite indicator.
2000a, EPA, 2004), robust, model-free techniques for sensitivity analysis should be used for non linear models.
Variance-based techniques for sensitivity analysis are model free and display additional properties convenient for the present analysis:
and to explain 92 they allow for a sensitivity analysis whereby uncertain input factors are treated in groups instead of individually they can be justified in terms of rigorous settings for sensitivity analysis,
as we shall discuss later in this section. How do we compute a variance based sensitivity measure for a given input factor i X?
The usefulness of I s, Ti S, also for the case of non-independent input factors, is linked also to their interpretation in terms of settings for sensitivity analysis.
we show in Figure 7. 2 a sensitivity analysis based on the first order indices calculated using the method of Sobol'(1993) in its improved version due to Saltelli (2002).
Rep. of Variance of country rank Non-additive Expert selection Weighting Scheme Aggregation System Exclusion/Inclusion Normalisation Figure 7. 2. Sensitivity analysis results
, Rep. of Total effect sensitivity index Expert Weighting Aggregation Exclusion/Inclusion Normalisation Figure 7. 3. Sensitivity analysis results based on the total effect indices.
The sensitivity analysis results for the average shift in 100 ranking output variable (Equation 7. 2) is shown in Table 7. 2. Interactions are now between expert selection and weighing,
there is not much hope that a robust index will emerge, not even by the best provision of uncertainty and sensitivity analysis.
Robustness and sensitivity The iterative use of uncertainty and sensitivity analysis during the development of a composite indicator can contribute to its well-structuring.
Uncertainty and sensitivity analysis are suggested the tools for coping with uncertainty and ambiguity in a more transparent and defensible fashion.
In Sensitivity analysis (eds A. Saltelli, K. Chan, M. Scott) pp. 167-197. New york: John Wiley & Sons. 12.
Homma, T. and Saltelli, A. 1996) Importance measures in global sensitivity analysis of model output. Reliability Engineering and System Safety, 52 (1), 1-17.60.
Saltelli, A.,Chan, K. and Scott, M. 2000a) Sensitivity analysis, Probability and Statistics series, New york: John Wiley & Sons. 123.
Saltelli, A.,Tarantola, S. and Campolongo, F. 2000b) Sensitivity analysis as an ingredient of modelling. Statistical Science, 15,377-395.125.
Saltelli, A.,Tarantola, S.,Campolongo, F. and Ratto, M. 2004) Sensitivity analysis in practice, a guide to assessing scientific models.
A software for sensitivity analysis is available at http://www. jrc. cec. eu. int/uasa/prj-sa-soft. asp. 126.
Sobol',I. M. 1993) Sensitivity analysis for nonlinear mathematical models. Mathematical Modelling & Computational Experiment 1, 407-414.130.
Tarantola, S.,Jesinghaus, J. and Puolamaa, M. 2000) Global sensitivity analysis: a quality assurance tool in environmental policy modelling.
In Sensitivity analysis (eds A. Saltelli, K. Chan, M. Scott) pp. 385-397. New york: John Wiley & Sons. 136.
ICT and e-Business Impact in the Transport and Logistics Services Industry.pdf
A Sensitivity analysis, Working Paper No. 2005-20, Federal reserve bank of Atlanta, August 2005. Porter, M. 1980.
INNOVATION AND SMEs ITALY.pdf
a Sensitivity analysis, Economics of Innovation and New Technology. Vol. 15 (4/5), pp. 317-344.
JRC85356.pdf
30 4. 3 Sensitivity analysis...30 5 Data Sources...32 5. 1 QS WORLD UNIVERSITY RANKINGS by QS...32 5. 2 FP7 database by EC DG Connect...
in order to present EIPE CI on a scale from 0 to 100, the values are standardized with the Minimax procedure. 4. 3 Sensitivity analysis An important issue related to the construction of composite indicators is weighting.
a sensitivity analysis is applied. Sensitivity analysis is the study of how the uncertainty in the output of a model can be apportioned to different sources of uncertainty in the model input (Saltelli, Tarantola,
& Campolongo, 2000). 31 The weightage allocated to each sub-indicator is varied by between the three sub-indices in the following way:
Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. Journal of the Royal Statistical Society Series A, 168 (2), 307-323.
Sensitivity analysis as an Ingredient of Modeling. Statistical Science, 15 (4), 377-395. Spizzirri, L. 2011.
MIS2014_without_Annex_4.pdf
and the results of the sensitivity analysis. 1. Indicators included in the IDI The selection of indicators was based on certain criteria,
(which tops the IDI 2013). 6. Sensitivity analysis Sensitivity analysis was carried out to investigate the robustness of the index results,
The future internet.pdf
and configurations for managing services and networks are used to ensure transference of results to other systems as result of sensitivity analysis.
< Back - Next >
Overtext Web Module V3.0 Alpha
Copyright Semantic-Knowledge, 1994-2011
