The MCM Process

Contents

Introduction

Multicriteria Mapping (MCM) was first proposed in 1997 (3). It derives from the most prominent of a wide variety of decision support tools developed in the field of multicriteria assessment (1, 2, 3). As such, it is based on long-standing and firmly grounded principles and disciplines that have been explored and tested over many decades in this area (4).

However, MCM also takes account of some well-established criticisms of multicriteria approaches (5). In particular, it is also informed by a rich literature in the field of 'science and technology' studies. This reveals the intrinisc problems in 'aggregating' different social views. It highlights the importance of various kinds of uncertainty - and of understanding how pictures of 'best' (or 'worst') choices often depend on the way the issues are 'framed' (8)).

It is for these reasons that MCM resists uncritical use, as a way to justify some single, apparently simple, 'best' choice. Ironically, this means it can be more straightforward to use than other multicriteria methods. This is because there is no attempt to claim some uniquely complete or definitive status for any particular view.

Yet, though straightforward, MCM can also be broader in scope than many other methods. This is because it allows inclusion of radically different data or viewpoints, without imposing undue constraints or assumptions. It also aims to go beyond narrow quantitative analysis, to give equal attention to qualitative factors and issues of principle. As a result, MCM may (if used properly) help to 'open up' social appraisal - allowing greater scope for expressing and exploring perspectives that may otherwise find themselves marginalised.

A four-step process

In common with other multicriteria approaches, MCM has four basic steps:

1. Develop a set of 'options'

2. Characterise a range of 'criteria'

3. 'Score' each option under each criterion

4. Assign a 'weight' to each criterion

One end product of these four steps is the calculation of an overall 'rank', expressing - for the viewpoint in question - the relative performance of each option under all the criteria taken together. Here, MCM follows the well-established linear additive weighting procedure, in which the rank simply represents the weighted sum of normalized scores. Participants are free to cycle through the four steps as much as necessary, to arrive at a final picture with which they are satisfied.

The following diagram shows the four steps, and includes an example of the 'floating bar' charts we use to show the range of the ranks calculated for a participant.

At each stage, MCM enables the collection of detailed information concerning the reasons why participants hold different judgements. This is done by providing many opportunities and cues for inserting text notes into the software. MCM sessions should also be audio-recorded and transcribed, so that key excerpts can be combined in later analysis with other relevant background information to give a more rich and robust qualitative picture.

How MCM is different

Unlike most other comparable approaches – both in the field of decision analysis and more widely – MCM focuses more on 'opening up' than on 'closing down' a decision or policy process (6, 7). Researchers gain a systematic picture of the precise ways in which different perspectives vary on the issues and options in question - as well as their practical implications for appraisal.

Although MCM can be used – like other assessment methods – to identify an overall ‘average’ picture across diverse perspectives, the distinguishing value lies in the detailed and systematic picture of differences. This involves compilation of a rich and highly structured body of quantitative and qualitative information concerning the detailed framings associated with individual perspectives. By contrast with other social elicitation methods, MCM retains a central focus on the concrete implications for the relative performance of different strategic or policy options.

MCM also spans a divide between quantiative and qualitiative. The authority and clarity of quantitative methods can directly address policy makers need to justify decisions. But these can be narrow in scope, suppress uncertainties and be insensitive to wider considerations and contending views. Broader qualitative approaches may accommodate more diverse perspectives, issues and uncertainties, but can have difficulty focusing on specific decision contexts and  providing political justification. Both quantitative and qualitative methods can often be quite restrictive and highly reliant on the interpretations of analysts. In response, MCM seeks to combine qualitative and quantiative inputs, so as to focus directly on key policy questions, whilst allowing maximum scope for participants themselves to shape and validate the overall picture that is produced of their own perspective.

Particular features of MCM that support this unusual combination include:

  1. A core set of diverse options are precisely defined in advance by the research team for purposes of focus and comparison. They may also define a set of discretionary options, which can be rigorously compared across those who wish to consider them. Crucially, however, participants are free also to redefine these predefined options or add additional options of their own.
  2. Participants are entirely free to choose and define their own criteria rather than having these imposed upon them. Unlike some approaches, these do not need to be assembled in advance by analysts into some single rigid ‘value tree’. This flexibility does not affect the comparability of the final results, since these are defined in terms of the 'performance' of policy options.
  3. Careful attention is given in scoring to the exploring and documenting of uncertainties: the way in which performance may vary for any individual participant, depending on assumptions or context.
  4. A clear picture is given of performance under each individual viewpoint, which is validated during the session by the individuals concerned.
  5. In later analysis, options, criteria, scores, uncertainties and ranks can all be aggregated in various ways across groups of participants or all participants taken together. This can be used to explore various dimensions of the commonalities and differences between different perspectives. Although aggregate conclusions can be drawn in this way, the primary focus lies not on generating an artificial consensus, but on mapping the framing conditions that underlie these.

MCM provides a picture from each viewpoint

By combining a tight focus on decision or policy options while at the same time 'opening up' the practical implications of different real-world perspectives, MCM tries to avoid a serious but often neglected problem suffered by economic, decision and risk assessment techniques, as well as by many more qualitative deliberative and participatory approaches. This problem concerns the way in which such methods attempt to derive a single definitive picture of option performance irrespective of the divergent uncertainties, interests, priorities, and values associated with different expert and socio-political perspectives.Where they are used in this manner to 'close down' policy debates, such methods can undermine underlying principles equally of 'rationality' or 'inclusion'.

To the extent that it resists such attempts at closing down, MCM can adopt the most straightforward of theoretically valid mathematical procedures used in decision analysis, thus enhancing the important qualities of accessibility (to participants) and transparency (to third parties).

MCM has been used in a wide variety of contexts

You can read more about past and current users of MCM.

Ten principles of Multicriteria Mapping

In order for an appraisal project legitimately to be referred to as a Multicriteria Mapping exercise, it should comply with the ten basic principles listed in this document:

References

  1. Keeney R, Raiffa H, Meyer R. Decisions with Multiple Objectives: Preferences and Value Trade-Offs. John Wiley: New York, 1976.
  2. Von Winterfeldt D, Edwards W. Decision Analysis and Behavioural Research. Cambridge University Press: Cambridge, 1986.
  3. Stirling A. Multi-criteria mapping: mitigating the problems of environmental valuation. In: Foster J (ed.). Valuing Nature. Routledge: London, 1997.
  4. McDowall W, Eames M. Towards a Sustainable Hydrogen Economy: A Multi-criteria Mapping of the UKSHEC Hydrogen Futures. Policy Studies Institute: London, 2006.
  5. Dodgson J, Spackman M, Pearman A, Phillips L. Multi-criteria Analysis: A Manual. Department of the Environment, Transport and the Regions, The Stationery Office: London, 2000.
  6. Stirling A. Analysis, Participation and Power: justification and closure in participatory multi-criteria appraisal. Land Use Policy 2006; 23: 95–107.
  7. Stirling A. Opening Up and Closing Down: power, participation and pluralism in the social appraisal of technology, Science Technology and Human Values, 33(2), 262-294, March 2008.
  8. Stirling A. Science, Precaution and the Politics of Technological Risk: converging implications in evolutionary and social scientific perspectives, Proceedings of the New York Academy of Sciences, 1128, 95-110, April 2008