Blog

Avoid this trap

We all use tests to help us make decisions. We take a test, and based on the results, we make a decision. A common mistake occurs when we lose sight of the fact that there are two types of tests in terms of purpose:

1. Tests meant to choose the best option for our needs;
2. Tests meant to simplify the choice and eliminate options.

One could argue that mathematically, these are the same, but pragmatically, they are very different. Even more problematic is when we conduct tests to eliminate options to simplify decision-making and later use them as if they were meant to choose the best option for our purpose.

Tests for selecting candidates

An example would be the tests given to choose candidates for certain positions in the job market, for which candidates have to pay to practice or where there is no transparency regarding the type of questions or the grading criteria used for evaluation (for those surprised by this idea, I remind you that hiring tests rarely have grading systems communicated in advance in the private sector). There is a vast market of tests that, by design, aim to narrow down the number of candidates for the decision-maker and generate profits for the test authors. No problem here, as everyone has the right to pursue their own goals. Problems arise when these types of tests are used to argue that the selected individuals are the best suited for the position. If you have to pay to prepare for a test to enroll in a university, course, or job competition for a position at a company, this is a test designed to make the selection easier for the decision-maker, under the guise of a skills test. If everyone can prepare for the test based on their own intellectual/physical/skill and resources, then it is a test meant to select those with the most suitable capabilities. If barriers, like payments or lack of access in information prevents some people from preparing for the test, then the purpose of the test is not to select the best, but to ease the choice for the decision-maker. Unfortunately, personal financial resources are not a good indicator of merit for anything other than the ability to make money in the current societal context. It is certainly a skill that not everyone has, but it cannot be an indicator for anything beyond itself.

Tests for evaluating researchers

Another example would be the development of scientific journal excellence standards based on quartiles and the evaluation of researchers based on the quartile in which the journals they publish in are. This is a problem because scientific journals which aim to be top ranked use tests designed to narrow down the number of accepted articles (because they are positively evaluated if they have a very low acceptance rate) rather than select articles that contribute to science. As long as articles are rejected because they are not interesting to the journal’s audience (which is different from a problem that does not fit the journal’s topic), that journal strays from the objective of selecting articles with scientific results within its field and leans towards profit and audience goals, which are more suited to commercial journals without scientific excellence goals. Furthermore, as long as a researcher’s development depends on the funds of the institution in which they develop and the opportunities available during their development, and as long as certain countries, fields of science, research institutions, and universities have access to very large research funds while others have very little, researcher evaluation must take these inequalities into account, using systems of publishing scientific results based on contributions to science rather than the need to reduce the acceptance rate of articles in highly ranked journals.

Procedural solutions and their limits

For decision-makers

As a decision-maker, when we outsource to a decision consultancy and accept certain tests that it proposes, we have the responsibility to check a few things if we want to select the most suitable options/people and accept that these options/people can have any background and come from any environment:

• What kind of options/candidates have access to those tests;
• What kinds of barriers they might face in order to perform well.

This is where values become important for our decision: Is it important for us to accept that some very suitable candidates or options can come from any environment/context, or not?

Some decision-makers argue that many resources are needed to select the most suitable candidates, but this is not correct. This is at least a false problem and, at most, one meant to avoid the effort of self-awareness regarding organizational needs. A deep understanding of one’s own needs and the ability to request exactly what is necessary for that position is required. In fact, many organizations do not know very well the criteria needed to perform certain job tasks and believe that doing what everyone else does will bring them the success they see in others. In reality, if we look closely, how many managers and recruiters know exactly what is needed to perform the tasks of a particular position? Generally, very few, especially those who have never performed those tasks themselves. Additionally, we must accept that sometimes we will simply use the first viable option from many that are suitable. It is not always necessary to choose the best version but the first that meets our criteria (the “satisficing” criterion). However, in these cases, transparency is a decision-making value that may be essential to the context in which we make the choice, especially if it involves decisions with ethical implications, such as hiring personnel or evaluating researchers, as discussed earlier.

For organization and institution designers

For situations where decision-makers do not know the necessary details for evaluation, various institutional mechanisms have been developed. For example, in the scientific evaluation of articles for publication, the institution of referees has been developed. The editor, not being specialized in all areas of a subject, even one as narrow as those around which scientific journals are built, seeks the advice of a specialist in the narrow field of an article. To avoid conflicts of interest, neither editors, nor referees are paid to give a verdict. This system worked as long as referees were financially independent and had control over how they allocated their research time and, moreover, shared the ethical values of the scientific community. However, today referees do not always meet these first two conditions, making it increasingly difficult for editors to find referees specialized in the field of an article who also have financial independence and control over their daily tasks to dedicate the time needed to evaluate an article.

As in the case of decisions about allocating the most suitable candidates to vacant positions, in this case, the necessary information for making decisions that lead us to select the best researchers for their positions is closely tied to knowing the research work in detail when making decisions about evaluating and choosing excellent researchers: a deep understanding of the skills needed to perform academic research in each specific field and a clear definition adapted to the context of each research field. These are things that would guide decision-makers toward identifying excellence in research. For now, the level of understanding of these issues remains unexplored as many studies on performance in research focus on evaluating results from tests meant to make choices easier, not to select those who contribute to scientific development.

Limits and implications of solutions

You will notice that in the two specific cases discussed, I did not mention “the best” because this is always a theoretical measure relative to the chosen goals. In concrete situations, we always need to discuss the particular goal. The goal of research is the advancement of knowledge, while the goal of personnel selection is to identify individuals who can perform job tasks while optimizing the goals of the organization that hires them.

It should also be noted that in this case, we are not discussing a meritocratic system but one oriented toward achieving a pragmatic goal. When the goal is clearly chosen, from an organizational design perspective, we can talk about designing meritocratic institutions or not. Additionally, a meritocratic institution is not related to the idea of an individual who “deserves” to become or develop in a particular direction. Meritocratic systems do not necessarily reward potential, but results, even in the early stages of an individual’s development, when, pragmatically, it would be unlikely to have results. For those wondering what a non-meritocratic system oriented toward achieving a goal looks like, an example is systems that select candidates based on potential and then provide them with resources to achieve the goal, sometimes rewarding the ability to reach the goals with minimal resources, other times rewarding reaching the goal with the given resources. Of course, there are other such examples of institutional design.

In conclusion, the use of the two types of tests (those oriented toward simplifying the decision through elimination and those oriented toward identifying the optimal option) is always justified in the right context. A major error is to use a simplification test and claim that it yields the optimal solution. Decision-makers must always be aware of what kind of test they are using, even when these services are outsourced, or especially in this case. And individuals who, for various reasons, are subject to an evaluation system, must distinguish what type of test they are entering to avoid using it for self-assessment when it is not appropriate.

Decision counseling as a solution

Experience and knowledge in decision-making are also essential to understanding the implications of the type of decision you are using to make choices. If you are facing difficult decisions as an individual, entrepreneur, manager, or employee, a specialized decision counselor can help you choose what serves all your goals, not just one. A specialized decision counselor will never tell you that your goals are incompatible but will support you in refining and finding the most appropriate way to achieve all the goals you genuinely pursue. Sign up for a decision counseling session (https://decisionresearch.eu/) to understand what decision method is suitable for your context. Sometimes, new errors or difficulties you thought you had overcome may hide in the complex details of the particular decision-making situation you are in. Your decision counselor can help you save time in identifying these and adopting the most suitable decision-making approach.

When social class meddles with your decisions it may “feel” like you are lacking fundamental traits. On a closer look, it would ease your mind and sharpen your strategies to differentiate when you’re swimming against a social phenomenon or against your own ghosts.

Social class is said to have been abolished in many countries when socialists and communists dismantled the bourgeoisie. But communism only invented a different kind of class, the nomenclature and declared that the proletariat was to be worshiped instead of the intellectuals and previously rich, but never more than the party. After the fall of communism, capitalism took back its dominant place and with it, the idea that anybody who is enough of something – smart, astute, innovative, organized, informed or simply has a lot of money – could do and be anything. Did these political and economic systems and their ideologies truly dismantle social class? Absolutely not.

All they did was to conceal it and make it a taboo topic of discussion. If your parents are rich you are said to be able to drive the business you inherit to failure or success, if you are poor you are said to have the opportunity to become anything you want – it is all up to you, as most self-help books keep telling us. My argument here is that social class is definitely not dead and it matters more than you think. Should it stop you from trying to get the things that you want to achieve? Not at all. However, it may help you understand why it takes longer, why it takes more effort, why you need to work more than others, why opportunities don’t spring up like mushrooms after a rainy day and why your group of friends cannot help you with anything even if they want to or why they can make all the difference in the world. It may also help you calm down that anxious feeling that you are not doing things fast enough, well enough and why you cannot seem to secure the social capital needed to get you ahead. Social class comes with the social capital and a resourceful one. Also, it might help you understand that you may be higher social class in your country of residence, but as a migrant, your status changes, as social capital amenable to class is international only at the highest levels.

But first, let’s understand what social class is. Generally, in sociology, social class is a concept that is measured by using three main variables: income level, education level and occupation type. Funny that the very things that we are told to be available to everyone, according to their own effort, strength or wits, are actually the stuff that makes social class possible. In more detailed analyses of social class these three defining variables need to be backed up by the same ones measured for a person’s parents. In other words, if you are the first generation to experience social class mobility, it is only your children that will experience some fast and notable benefits that would improve their chances.

If you come from a family which is already well educated at a formal level, has some higher level income, but does not necessarily occupy a white collar type of job, you may already be on the higher end of life as long as you do not get into dramatic trouble. But if your parents come from lower income and blue collar or even a physical kind of labour, even if they have some higher education, your options in life may be a bit restricted, at least in certain types of jobs. While it is true that it is possible to reach a high income, get higher education and have a white collar job yourself, even if your parents don’t, the probability of this happening may not be as big as for those whose parents were already higher up the social ladder and may require more effort. This may sound obvious in theory, but the way it feels when you are deciding where to go with your life, may feel tremendously different. And if on top of this you are a woman, your odds are against you anyway. I know we have positive discrimination policies in place and that we now value women in science and so on, but we still have more men in higher level positions than women in so many fields, and gendered industries and so on and so forth. We have made progress, but there is still some more to go.

I know that most of you know this in theory and understand the concepts. But do you really understand how it applies to you and why this matters when you make career or life decisions or when you evaluate your efforts or, for that matter, when you are evaluated at work?

First of all, those who either by family of origin or by current status or both classify as not higher class may take longer to achieve their dreams, may need to work harder than those who fit the higher class bill and may even achieve less things, have less opportunities and fail more opportunities because of not having access to the right information at the right time. Their route from where they are to where everybody wants to be is actually longer.  If your parents live in Palo Alto California, the steps you need to take in order to get into Stanford University are much less in number than the steps you need to get there from Kenya or even Albania, both physically and socially speaking. Everything will stand in your way or facilitate your path, from culture, to type of education, school you go to, neighbourhood, family structure and resources (e.g. if you were raised by your grandparents, by a single parent or all your family, if your parents suffered from some addiction or if they could help you with your math homework, etc.) to the friends who live next door to you and so on. Clearly, your personality, hard work, motivation, drive and so on will play a part, but it will not be the most part. Ceteris paribus your individual traits will make a difference, but how many times have you been in a job interview which evaluated people against their background and the difficulties they had to face in life? Absolutely none.  And would that be feasible for employers? For sure, where there is a will there is a way. Would it lead to any gains for employers? I guess it depends on their goals.

Now, this is not an argument against those born in wealthy families. They have their own problems and difficulties. In the end, no life is devoid of tragedy, loss but also immense joy. Some more than others. However, the main argument is that when you make decisions about your life and especially when you allow others to evaluate your work and your results according to some socially agreed grid, you need to understand where you came from  in order to see where you are now and how much of your route you’ve already walked. You need to understand why you might miss opportunities, why things “don’t seem to happen to you” and why you constantly seem to fall short of something that others have. These may all be symptoms of social class and you need to look closely and diagnose the problem sharply. You will gain peace of mind, clarity and possibly a chance to be kinder to yourself and to others.