The telephone as a market research tool? Well established for decades as a reliable and fast instrument for high quality surveys. And the Internet? Just a few years after it became an everyday tool for consumers in the early 2000s, it complemented traditional survey standards – for tech-savvy audiences who were either reluctant or difficult to reach by phone, and who wanted to control the moment in time to take part in the survey and in which pace to complete it.
Surveys by telephone – Computer Assisted Telephone Interview (CATI) and by web – Computer Assisted Web Interview (CAWI) have therefore been recognised for a long time and both have their advantages when it comes to researching specific target groups, products or opinions. Both methods are ideal for topics that require conducting surveys abroad.
However, there can be methodological blind spots in both telephone and online research.
The answer to declining participation and heterogeneous target groups: Mixed mode
It’s no secret that the basic willingness of all target groups to participate in surveys has been declining for years. It is a problem that we market research institutes are confronted with on a daily basis. This is particularly true for small niche samples where representativeness can only be achieved with considerable effort. In addition, clients are increasingly reducing their budgets, but still expect consistent quality and, ideally, increased speed.
In partnership with moweb research, AMR Advanced Market Research presents an innovative, cross-method solution that adresses the challenges mentioned above: Mixed Mode Research.
Mixed mode refers to the simultaneous use of telephone and online surveys within a single market research project. The aim is to combine the advantages of each method and unite the best of both worlds in order to generate greater reach, better sample utilisation and even higher data quality.
This applies to both niche target groups and to particularly heterogeneous groups. In this cases it is challenging and requires additional effort to achieve a representative sample using a single method.
With mixed mode, participants are given a choice – and thus, are more likely to take part.
Mixed Mode works with the parallel or sequential use of online and telephone surveys. We give our participants a choice: either the personal telephone survey, conducted by an interviewer, or the independent completion of a questionnaire on a digital device. In some cases, it might be advantageous to use both options.
Not only do we provide a choice of method, but we also allow respondents to change mediums during a survey, or to continue or complete the process at a later time using their preferred method. Respondents appreciate this flexible choice as a service, which can have a positive impact on their willingness to participate.
CATI follow-ups allow us to remind participants by telephone to (re)start a survey after they have been recruited online. By ‘CAWI follow-ups’ we mean reminders by email if the telephone contact was unsuccessful – with a request or offer to call back.
Mixed Mode reaches heterogeneous target groups. Example: A school
A school is probably not the first project that comes to mind when you think of challenging and heterogeneous target groups. However, our client set us the task of surveying three target groups from a specific type of school in a specific state at very specific grade levels.
The subject of the survey was the teaching material used in classrooms in a particular subject canon, in detail the opinion on their selection and the evaluation of the material. A representative group of parents, pupils and teachers should be interviewed, each with their own questionnaire.
Structurally, pupils, parents and teachers are far from homogeneous. Pupils, and thus young people, are very comfortable with the Internet. This is only true to a very limited extent for their teachers, and to some extent also for their parents. Parents and teachers are therefore much more difficult to reach via digital media than pupils. In terms of numbers, teachers are considerably smaller than the other two groups.
Mixed mode increases utilisation rate & representativeness – for more quality
All three target groups are therefore clearly defined and distinguishable, but very heterogeneous in terms of sample size, media affinity and accessibility. These different survey preferences therefore precluded an ‘online-only’ (CAWI) approach. However, a pure telephone survey was also out of the question as the client’s budget was too low.
At this point, we therefore used a mixed mode of CATI and CAWI in a three-stage process:
- Addressing all three groups via digital media (CAWI).
- Approaching teachers who cannot be reached online by telephone (CATI).
- Completion of the field phase with a minimum sample of parents and pupils via CATI to compare and verify the responses collected previously.
That’s how we achieved the necessary representativeness of all three subgroups within the budget and time constraints, and provided our client with an even higher response rate and data quality in the third field phase.
Mixed-mode surveys require a sensitive design
As explained, a combination of methods allows for successful market and opinion research, particularly when dealing with challenging target groups. However, this method presents certain challenges and demands specific technical, personnel, and organizational requirements.
For example, the data sets from two different survey sources (CATI & CAWI) can only be coherently integrated, analysed and processed if they are compatible with each other. This requires sophisticated data management using appropriate software solutions.
Identifying and eliminating duplicates and multiple respondents is also key to obtaining valid survey results. However, fraudulent respondents also attempt to express their opinions multiple times in a survey using different survey forms. We effectively counter this challenge with sophisticated respondent tracking and data de-duplication.
Biases as a challenge & harmonisation of questionnaires
Another sensitive issue in all forms of survey is the bias in respondents’ answers. Trained interviewers are aware of the well-known social desirability bias, i.e. the danger that respondents tend to give answers that are socially desirable and thus deviate from their true opinions. Such a bias (there are many more – here is a brief overview) can be avoided or at least reduced by the design of the questionnaire itself.
Compared to a telephone survey, an online survey is less likely to suffer from social desirability bias, as the anonymity of the internet and the fact that participants are unobserved usually frees them from social pressure.
However, one thing is clear: bias plays a role in both face-to-face interviews and impersonal online surveys, but the characteristics of each are different. Mixed mode gives us the opportunity to balance the disadvantages of one method with the advantages of the other. Ultimately, the whole survey has to work without ending up with questionnaires that are too different. However, we can make sure that the answers can be merged later through a well-coordinated and comprehensive design.
Mixed mode for a better respondent experience
Mixed Mode surveys require a high degree of experience and expertise in planning, implementation and evaluation. AMR Advanced Market Research and moweb research are well established institutes and are both leading providers of telephone (AMR) and digital online surveys (moweb). Through the joint management of both companies, we not only combine the relevant expertise of professional market researchers, interviewers and programmers, but have also brought the interface management for the mixed-mode process described above to market maturity.
Surveys with highly heterogeneous target groups, each of which has to be addressed with different media, require new approaches to achieve valid, fast and cost-effective results. Mixed mode offers exactly that: high utilisation and response rates when individual survey methods reach their quantitative and qualitative limits. Not only does this approach compensate for the weaknesses of individual modes, it also results in a better respondent experience.
Mixed mode for increased data quality
At a time when budgets are shrinking and expectations are rising, it is vital that you remain committed to delivering the best possible results for your business customers. This is achieved by continuously measuring, deploying, and refining our resources and expertise as market and opinion researchers.
This is also essential, particularly as society, markets, and technical contexts (AI being a crucial keyword) are becoming increasingly complex and demand new solutions.
In this context, mixed mode is another product innovation to address this trend and meet our customers’ needs in the future.
