Awareness Surveys: The Data-Driven Way to Read People’s Minds

Sending a satellite around the Earth and engineering submarines to conquer the sea’s depths are among the most daunting challenges taken up by humans. We have performed fairly well at both. But then there is an area that seems even more difficult to explore — the human mind. The mind continues to intrigue professionals, who find it most challenging to measure human perceptions, beliefs, and awareness. These are undoubtedly among the toughest parameters to measure.

Deriving meaningful insights from people’s knowledge and understanding plays a major role in policy making and planning for social development. However, not much has been written about how to do this.

In the article that follows, we explore why awareness testing is so important and various ways to get make awareness surveys more accurate. The key to all of this is understanding how the mind answers about what it knows.

What does it mean to be aware?

Awareness is the perception of and cognitive reaction to a particular situation or event. We come across this term very often. From self-awareness to less spiritual concepts like brand awareness, man has become obsessed with the perception of our surroundings and ourselves.

From self-awareness to less spiritual concepts like brand awareness, man has become obsessed with the perception of our surroundings and ourselves.

Gaining awareness is a complex process. Introducing a concept or a fact to someone is only the first step. This fact either gets pushed out of the person’s mind or gets registered in his psyche as knowledge. The last and the most significant step to gaining awareness is when someone uses this knowledge to bring about an attitudinal or behavioral change. This is when the person is finally productively aware of the concept or fact.

Why test awareness?

Awareness testing is always an important first step. In many cases, awareness testing can help to identify the cause of a problem. When the problem is not caused by low awareness levels, awareness testing is still useful to prove that awareness campaigns had been successful and to zero in on the actual cause of the problem.

In its HIV-AIDS campaign, the Government of India works to reduce the number of new HIV-AIDS cases each year through awareness programs like television commercials. But these programs aren’t the most important part of the campaign. The real work starts months after these commercials go on air, when the government must test the effects of the campaigns. How much do people know? How has it affected their practices? If it hasn’t, why has it not?

Take the example of a remote village with a high number of new HIV-AIDS cases each year. In this village, the awareness survey has shown that people are aware of HIV-AIDS transmission and risk. However, cases are still going up. Why? This village’s HIV-AIDS rate is driven by other factors than a lack of awareness — the jobs and attitudes of the community’s males, and the economic dependency of women on their husbands. The males’ migrant jobs put them at higher risk of contracting HIV and bringing it back to their village. The women are aware that their husbands were potential carriers of the infection, which increased the women’s risk of contracting HIV. However, women in this village are unable to say no to these unsafe sex practices due to their socio-economic status.

In the example above, an awareness survey disproved the standard belief — that new cases are caused by a lack of awareness.

In the example above, an awareness survey disproved the standard belief — that new cases are caused by a lack of awareness. Instead, testing showed that the government’s campaigns had been successful, and that a lack of awareness was not the cause of new HIV-AIDS cases. By doing awareness testing, the government was able to zero in on the actual cause of new HIV-AIDS cases. The government could then amend its efforts for that village — from intensive awareness campaigns alone to specific behavior and practice changes, which could include counseling, behavior modification, and female empowerment schemes.

Understanding the importance of such awareness testing, the government of India has ensured that its major surveys contain sections to measure people’s awareness. A major portion of the National Family Health Surveys includes questions about people’s awareness of health issues and healthy practices, how they gather this knowledge, and their reasons for behaving with or against their knowledge. These questions provide a comprehensive pool of information to policy makers who can then better target their health policies.

Awareness surveys are valuable but hard to carry out. There are 3 main aspects that need careful attention — designing good questions, preventing guessing, doing cognitive pre-testing, and conducting a field trial.

Designing an awareness survey

An awareness survey’s result largely depends on how clear the surveyor is about what they want to know. Just like in any scientific study, the research question needs to be precise.

The first step of awareness testing is determining exactly what types or levels of awareness the survey should test. For example, an NGO that distributes mosquito nets to reduce malaria cases could be interested in measuring two things: people’s knowledge on how malaria spreads, and the number of people using these nets. Zeroing in on these aims is essential to writing better questions in awareness testing.

When testing awareness, it is also essential to understand the importance of getting a holistic picture. Knowing where and how people gain information, understanding people’s preset beliefs that can get in the way of accepting new information, and learning about the traditions or difficulties that keep people from implementing behavioral changes are important aspects to consider while designing awareness survey questions.

Preventing people from guessing

A major problem with awareness surveys is that people tend to make guesses to appear to knowledgeable. According to psychologists, this stems from the fact that not knowing something is looked down upon socially. Hence, participants tend to guess from the available options to seem more knowledgeable. There are many ways to reduce this likelihood — framing the questions in the right fashion, giving a good introduction, and choosing the right number of appropriately worded options.

Giving context in an introduction

Questions 719 to 22 of the NFHS questionnaire for women test knowledge and awareness of TB. The questions ask “Have you ever heard of an illness called Tuberculosis or TB?” and “Can TB be cured” The options for the first question are “Yes” and “No”. The second question has an additional option of “Don’t know”.

There is a high likelihood that the respondent will say yes to the first question to seem knowledgeable, with the understanding that there are likely no consequences for lying. The next question then asks about the different ways in which TB spreads. Having lied at first with a yes, the respondent will feel compelled to guess how TB spreads. This leads to a chain of inaccurate answers.

An introduction might be useful at the start of this section, such as this:

“I am going to ask you a set of questions about a disease in order to understand your knowledge about it. This will help strengthen our awareness program. We expect that some people are not aware of one or more of the aspects of this disease. Please feel free to tell us if you do not know.”

In a lead-based damage awareness survey in the U.S., a similar introduction was shown to increase the likelihood of getting a “Don’t know” answer.

Framing it right

Besides giving an introduction, wording the options correctly is also important. Many times, respondents are more likely to choose “Not sure” over “Don’t know”. Asking for an opinion instead of asking for facts is also a good decision.

For example, consider the two given question and answer formats below:

• How does AIDS spread?
  • Sharing utensils
  • Coughing/sneezing
  • Sexual contact
  • More than one of the above
• Do you think AIDS spreads with sexual contact alone, or are there other factors involved?
  • Yes, sexual contact alone
  • No, other factors also
  • Not sure

A person who doesn’t know the answer is more likely to take a guess to seem knowledgeable for the first question than for the second. The second question motivates people who are not sure to just admit so. To get further details about the extent of knowledge or beliefs about the spread of HIV, the survey can ask an additional question to people who answer “No, other factors also”.

There are other factors that can play a crucial role in framing an awareness survey. These include the personality of the interviewer (non-intimidating, friendly surveyors are more likely to get honest answers) and his/her knowledge of the local language. Attention to such details can also help derive better answers.

When you word your survey options, don’t forget the MECE framework. Read more here.

Cognitive pre-testing

Cognitive pre-testing is another tool that can be used to design better awareness surveys. It can act like a dry run of the survey to get feedback. Conducted before the actual survey, it aims at collecting information from participants about how they interpret the questions, construct their answers, any difficulties they had in answering the questions, and other feedback. Data generated from cognitive pre-testing can be analyzed to re-design the survey for better responses.

Think-aloud method

One of the two common ways to design a cognitive pretesting interview is a think-aloud method. In this, the participant is asked to literally think aloud as he goes through the process of answering the survey. This method gives crucial information to refine the survey but can be tiresome for the participants.

Probing method

The probing method works slightly differently. It asks exploratory questions to the participant after he/she has answered each awareness question. Examples of probing questions include “How did you conclude this?”, “What do you understand about this question?”, and “What do you think this question wants to know?” Since probing questions are asked after the survey question, they do not interfere with the participant’s cognitive process.

Field trial

After the cognitive pretesting is done with, the awareness survey needs to be sent for a field trial. In a field trial, the survey is tested on people of different races, educational backgrounds, financial levels, genders, and age groups to understand which questions get higher “Don’t know” rates. Customizing the options and question types on the basis of these results is a great option if the survey is going to be administered on a small, easy-to-categorize audience. Analyzing the results of the field trial and further refining the survey is the last step before the surveys are ready for the public.

Creating an awareness survey can seem easy, but it’s actually a complicated procedure. To gain accurate, meaningful results, a lot of work needs to go into designing these surveys well. But, more than anything, awareness surveys can only be effective if we understand our respondents well enough and respect how their thinking works.