transparent choice Completing a Criteria Weights Survey Software User Guide
- June 3, 2024
- transparent choice
Table of Contents
transparent choice Completing a Criteria Weights Survey Software
Completing a Criteria Weights Survey
Overview
Why do we do it?
Not all your Criteria are of equal importance – Analytic Hierarchy Process
(AHP) enables you to quantify those different values
These are subjective judgement calls – by taking a survey & joining a follow
up session, you will help create the ‘wisdom of the team’
How do we do it?
Instead of direct allocation of weights, AHP uses Pairwise Comparisons to
weigh criteria This is because the human brain works better with comparing two
choices than it does with abstract values
What if there is disagreement between voters?
We will meet to discuss the results – and use the software to focus on the
points of disagreement. From there we can learn from one another, change
votes, or agree to differ and use an average.
How will this be used?
Each of our Sub-Criteria will have a Weight – these are multiplied by a score
for each Project for each sub-criteria… the results will be an overall
Prioritization Score for all prospective Projects
Navigation
-
Paste URL / Click the Link – Opens up a tab to complete your survey – note this is the
-
Survey Details – Shows the structure of the model & number of questions you need to complete
-
Criteria – Click to see detailed description (which you added when building the criteria)
TIP – you can add supporting documentation here -
Progress – Shows proportion of survey completed
-
Weight – Hit the button to start the survey
-
Join Meeting – Move into the ‘live’ meeting, once the Facilitator starts the session
-
My Votes – Analyse your responses once you have completed the pairwise review (see below for tips on this)
Pairwise Review
-
Navigate – Save progress, return to survey landing page & see progress so far
-
Navigate – Move within the Survey, selecting Criteria through the hierarchy, text, or by inconsistency (see below for details)
-
Question – Always appears in the format:
In the Context of (i.e. the criteria one level up) What is more important Criteria 1 or Criteria 2? -
Vote – Pick from 1 of 3 options: Criteria 1 / Criteria 2 or Equal using the Radio Buttons
-
Vote – Then (unless you vote ‘equal’) score the extent of the difference using the 9 point scale (AHP Best Practice)
-
Comment – Add key points to capture rationale behind vote – this is great to support discussion
-
Navigate – Move onto Next question or return to the top of the Survey page
Check Consistency
Inconsistency score is
a measure of how much your answers make logical sense compared to themselves
For example, if A>B and B>C but A<C this makes no sense – it’s logically
impossible
The software calculates this rate of ‘error’ & prompts you to review your
answers:
20% – Answers are too inconsistent: Please Review them & adjust some votes TIP
– think about reducing ‘Strong’ (or more) scores so they are less extreme
10-20% – Acceptability depends on the type of Criteria. For Strategic criteria
this level of inconsistency is OK. For more technical criteria this may be too
great. Align with your Survey administrator to agree.
<10% – Answers are fine. This is an acceptable level
View Weights
TIP – Once you have answered all questions, go to View Weights. Criteria
Weights represent the output of the algorithm that converts pairwise votes to
a normalized weight for AHP model criteria.
If they don’t pass your ‘gut check’ go back to your votes and double check
them: have you made a mistake, or been too extreme in one of your votes?
Iterate until you are happy with both the votes and the outcome.
- Drill into the Criteria – to move between high level & detailed view of outcome
- Decimal Places – change display of data
- Toggle View – To see scores for Global & Local Weights TIP – Local Weight shows how the sub-criteria compare to each other (they always add up to 100)
Global Weight shows the sub criteria share of the total model (calculated as Local Weight * Criteria Weight)