How to analyse advice from multiple people when making advice-based decisions?
Sections
00:00 – 02:25 Intro
02:26 – 04: 30 Tools
04:31 – 08:44 The decision making structure for data analysis
08:45 – 15:59 Getting from data to a decision-making structure
16:00 – 20:59 Using a decision structure for data analysis
21:00 – 28:00 Notes on aggregation and limits
Shortened Transcript
When advice-based decisions are part of the procedures in your company, it can be difficult to actually transform the various pieces of advice that one receives into a decision. Even though you might know a little something about analyzing qualitative data, this will not help you reach a decision. This is because analyzing data in order to make decisions requires a certain type of structure, as opposed to analyzing data from interviews or from focus groups aimed at exploring, understanding a process or uncovering meanings and so on. On the other hand, when the decision is complex, aggregating the data into a single “do this”, “don’t do that” framework can be a really difficult task.
In a well designed research aimed at making decisions, the structure of the analysis derives directly from the research design and is linked to each question. However, if your field is not research and you need to make an advice-based decision, you might want to steer your efforts more into implementing the solution for your own field of expertise, rather than understanding the nuts and bolts of decision-making research. So, this video is for you to take a short-cut and transform all the advice received into a decision.
In order to do this there are two issues to consider:
- The tools you can use to analyse and then aggregate the pieces of advice
- The decision-making structure needed to make a decision.
The tools.
If you can afford a simple tool for online team-work, I recommend using Miro to analyze the answers (possibly also for collecting the pieces of advice for either anonymous or nominal answers). This tool allows you to aggregate the data in a visual manner, copy, paste and cut any parts of interviewees’ answers and place them into your decision-making structure and thus cluster everything to a point at which you can make a decision. Miro has a team work feature, such that multiple teams can work in the same online workplace at the same time.
If, on the other hand, you would like to re-use a tool which you already own, for this new purpose, you can use Excel, LibreOffice or any available spreadsheet tool and you don’t have to spend extra money for this.
There are other specialized tools for qualitative data analysis, like NVivo, MaxQDA and RQDA packages in R, but these may require you to learn how to use these pieces of software and you will still need to structure your analysis to help the decision-making process. So, no matter what tool you are using, this video will be useful.
Right now, I will show you an example of work-flow that you can use in order to analyze the advice received and by structuring it such that it can help guide your decision. For this purpose we use decision structures.
What is a decision-making structure?
It is a construct – a.k.a. something that either you or a community or group makes – which links your cognition – i.e. what you know – to your behavior – what you do (Fig. 1). It is called a structure because its elements are linked to each other in certain ways. There are many decision-making structures available. The one I prefer to use is the most complete and it structures any piece of information designed to help you make a decision into the following categories: the decision problem, the options, the criteria, the comparison of the options based on criteria, the preferences between options, the relevant values, the emotions around performing the options and finally the identities which are influenced by performing the options.
Fig. 1 Decision Structures are sets of categories we use in order to link what we know to what we do.
Let’s take an example. Suppose that you work in an insurance company and you would like to take advice on changing the client referral policy. First of all you need to translate this question into a decision-making problem. For example: Should we change the client referral policy or not? Or Should we change the client referral policy using Policy strategy X or policy strategy Y? Once your question is defined (to simplify this video, I will assume that your question is your true question and well defined; in general for this there is a certain process to follow, but I will not get into this now), suppose you have initiated a short questionnaire or series of interviews to help you gather advice from relevant stakeholders in the company. And let’s assume that you did not know about decision structures before making this survey. You now have data that you need to aggregate in a way that helps you make a decision. It may be possible that some of your data is not usable for decision-making purposes. That can happen. But some of it might be. So, at this point you can start using the decision-making structure to analyze your responses. Here are a few questions to help you structure the data you have into information you can use to make a decision:
- How do your advisers frame the problem? Are there substantial differences?
- What do your advisers perceive to be the main goal behind this problem?
- What are the options they see? Do they mention other options than the ones you placed in your decision-making question?
- What are the criteria your advisers perceive to be important in order to achieve their goal?
- What are the most important values which are expressed throughout their answers?
- What are the emotions they express and what needs do they point out to?
- How do the options influence their perception of who they are and who they would like to be in the company? Are there any risks for their desired position as a result of implementing either option?
How can you use these questions?
Let’s see how you can do this.
To give an example of how an interview can be analysed by using this decision structure, I’ll present two ways: one is more visual, with Miro and the other more abstract, with Excel, but cheaper if you already use Excel.
Once you have all interviews analyzed in this way, all you need to do is decide how you’d like to aggregate them. Would you like to consider the most frequent answer (frequency)? Would you like to include everybody and find a solution to make each stakeholder happy (consensus)? Would you like to re-iterate the process with new solutions which have emerged from this round of advice?
I suggest that the way in which advice has been aggregated is always made clear and available to everybody involved. And keep in mind that anybody who challenges the way to aggregate has a valid question. With the same information, using different aggregation methods, you can reach very different decisions. So, be ready to justify your choice.
In this clip I have not touched on several important issues around advice taking: sampling the experts or stakeholders, defining the decision question, what are the other types of aggregation methods, how they are useful and under which conditions and also what other types of decision structures are out there and for what kinds of decisions are they useful.
I hope this helps and thank you for reading.
