A Heavyweight in Market Research: The Social Desirability Bias

»Do you find Germans inherently superior to other peoples?« – »Do you brush your teeth every night?« Extremely different questions, but they trigger the same reflex in surveys.

Let’s take the first statement regarding the superiority of the German people: it was only recently part of a representative study that citizens from the East German federal states were asked to answer – in writing and anonymously. Other questions in the same study – less polarising and rather innocuous – were asked of the participants personally by an interviewer.

Why this division into written-anonymous on the one hand and personal-direct on the other? The reason lies in the so-called bias of ‘social desirability’, which means that even völkisch-minded contemporaries would have answered an interviewer rather negatively to the question posed above.

»Social desirability effect or bias: respondents are more likely to express opinions and attitudes that they assume are in line with the social norms and values of society – and thus also those of the interviewer.« Source: Marktforschung.de

A small step back for background: humans are social creatures through and through. Almost everything that constitutes human behaviour is socially learned, i.e. FROM others or THROUGH others. People also like to feel they belong, at least when it comes to their ›own‹ group, regardless of whether it is a nationality, religion, age group or something as banal as a tennis club. Human and normal is also the constant judging of others and at least the belief that one is also constantly judged by others.

Not least because of such social dynamics, the average citizen knows quite well which behaviours are the norm in our society – and which tend not to be. What therefore meets with general approval and is thus socially desirable.

Social desirability bias in market and opinion research

AMR is an expert in telephone surveys and of course we want to find out what people really think or what they mean and how they act in every survey. However, this reality, by its very nature, rarely coincides with what society, at least officially, expects of its citizens in terms of goodness and morality. What happens when we as an institute ask questions about this contradiction?

Especially when it comes to socially sensitive topics – from political or religious attitudes to other personal matters such as smoking, alcohol consumption or behaviour in one’s own four walls – people are not always honest in surveys. Many know exactly what is considered socially acceptable and bend their answers accordingly. This means: they distort their statements in the sense of the aforementioned ‘social desirability’.

Of course, this bias is not new to AMR. We know reliably when and where this bias lurks, which factors influence it and which strategies and measures help to minimise or even avoid it.

Sensitive taboo topics need safe spaces in the survey

Socially sensitive or even ostracised topics, for example racism or discrimination, are a challenge not only in terms of content but also in the concrete interview situation. We regularly experience that people confirm their individual concern but are reluctant to openly share their concrete experiences with the problems.

It is possible that the respondents lack psychological security at this point, the subjectively felt ›safe space‹ to name their problems without fear, clearly and unambiguously. In the end, however, this information is missing from the survey results and distorts the objective social picture.

Similar patterns – i.e. no, embellished or evasive answers – are also found in questions about abuse or violence in the family. At this point, we must always ask ourselves as an institute: How do we obtain valid information despite such topics by providing answers that are as unbiased and meaningful as possible?

The bias in all elements of a survey

In AMR’s everyday work, the bias of social desirability is a constant companion, because it shows up in all phases of a survey: from the interviewer to the subjects, from the questionnaire to the method mix.

The interviewer: Professional with soft skills

Right at the beginning of every survey, building trust between interviewer and respondent is central, mainly by guaranteeing the latter anonymity and discretion. Although AMR is an established institute that stands for seriousness and competence in the industry, not every test person will answer personal questions absolutely truthfully.

If this is recognised in time, appropriate measures can be taken to counteract it. A proven way to do this: We explain to the participants all the essential information about the survey, e.g. the background and also – at least if allowed – the client. In addition, we explain the study design and the goals of the survey.

We regularly train our employees to conduct telephone interviews in an empathetic and trusting manner, even in tricky situations. Important soft skills in conducting conversations ensure that we always credibly convey to the counterpart the real interest in their person and statements.

Female or male interviewer?

Gender is not a factor to be underestimated in the disposition of interviewers for a particular study project. Depending on the topic, we pay attention to when men should speak with men, women with women and diverse with diverse – or quite deliberately mixed-gender. There are topics where this constellation and thus the critical relationship of trust cannot be left to chance and possibly only for this reason the participants do not want to or cannot speak openly.

The focus in each survey: The subjects

Apart from sensitive taboo topics where people prefer to run their statements through a social filter out of self-protection, fear or shame – what can be other reasons for socially biased because desired answers? It may well also be due to a lack of awareness of the great significance their statements will have for a socially important study, so that the relevance of their answers should always be emphasised.

Even in everyday surveys, where clients evaluate the quality of companies or brands, for example, professional interviewers recognise when the target group seemingly holds back with honest feedback out of politeness. Here, the motivating reminder that critical words do not offend anyone but are, on the contrary, desired helps, as only these will also help the client in a practical way.

Questionnaire: Gaining problematic insights indirectly

The questionnaire design also influences how target groups react to taboo questions. Instead of confronting them directly with a problematic issue in a telephone conversation – and thus possibly scaring them off – sensitive insights can also be gained indirectly. For example, a possible affinity to various drugs is not asked directly, but only hypothetically described. The target person is then only asked to assess how conceivable this scenario is for him or her.

Indirect questions or working with fictitious scenarios create more distance and leeway when answering. At least from the participant’s point of view, this reduces the subjective risk of discrediting themselves morally towards the interviewer.

The method: Mixed mode for ‘difficult’ topics

We have already seen that surveys do not have to be conducted mono-methodically per se in the example on ›German superiority‹. But such an approach – we at AMR call it ›mixed-mode‹ (link to the other text) – is anything but trivial, because it has, among other things, considerable effects on the design of two questionnaires as well as additionally on the complete evaluation process.

However, if the mix of topics requires supplementing the regular telephone interviews with an additional written survey (e.g. online via email), any effort is worthwhile due to the result of more valid and better data.

Conclusion

The ›social desirability‹ bias – like many other mental biases (see short interview below) – is quite coherent and understandable, but nevertheless forms a very complex phenomenon. It takes a lot of experience, psychological know-how and competence in developing survey designs to reduce the negative impact of one or more of these effects.

AMR has been gathering a wide variety of insights for its global clientele for a good 40 years now. For us, the telephone interviewers have always been at the centre of the entire service. Their quality and competence ensure that even with difficult, sensitive or tricky questions, no feel-good answers come to light, but rather the real truth and thus useful information.

So if the respondent at the beginning really does brush his teeth irregularly in the evening, his honest answer can be very valuable for a health education study. That is all that matters and not to expose a particular behaviour or non-behaviour.

---

Let’s ask Dr Satish Thalla, Key Account Manager at AMR

»A bias is a distortion or tendency in a human or machine decision-making process that results in an inaccurate or incomplete assessment. A bias can be conscious or unconscious and can be based on prejudice, misconception or other factors.« Source: Marktforschung.de