INFORMATION SHEET 2.

Questionnaire Design

Learning Objectives

After reading this Information Sheet, you must be able to:

· Understand the attributes of a well-designed questionnaire, and

· Adopt a framework for developing questionnaires.

Introduction
No survey can achieve success without a well-designed questionnaire. Unfortunately,
questionnaire design has no theoretical base to guide the marketing researcher in developing a
flawless questionnaire. All the researcher has to guide him/her is a lengthy list of do's and don'ts born
out of the experience of other researchers past and present. Hence, questionnaire design is more of
an art than a science.

Structure of The Chapter

A brief account of the key attributes of a sound questionnaire serves as the opening section of
the chapter. This is followed by a nine-point framework for developing an effective questionnaire.
These are the only two components of this chapter on questionnaire design.

The qualities of a good questionnaire

The design of a questionnaire will depend on whether the researcher wishes to collect
exploratory information (i.e. qualitative information for the purposes of better understanding or the
generation of hypotheses on a subject) or quantitative information (to test specific hypotheses that
have previously been generated).

Exploratory questionnaires: If the data to be collected is qualitative or is not to be statistically

evaluated, it may be that no formal questionnaire is needed. For example, in interviewing the female
head of the household to find out how decisions are made within the family when purchasing
breakfast foodstuffs, a formal questionnaire may restrict the discussion and prevent a full exploration
of the woman's views and processes. Instead one might prepare a brief guide, listing perhaps ten
major open-ended questions, with appropriate probes/prompts listed under each.

Formal standardised questionnaires: If the researcher is looking to test and quantify hypotheses
and the data is to be analysed statistically, a formal standardised questionnaire is designed. Such
questionnaires are generally characterised by:

· prescribed wording and order of questions, to ensure that each respondent receives the same
stimuli

· prescribed definitions or explanations for each question, to ensure interviewers handle questions
consistently and can answer respondents' requests for clarification if they occur

· prescribed response format, to enable rapid completion of the questionnaire during the interviewing
process.

Given the same task and the same hypotheses, six different people will probably come up
with six different questionnaires that differ widely in their choice of questions, line of questioning, use
of open-ended questions and length. There are no hard-and-fast rules about how to design a
questionnaire, but there are a number of points that can be borne in mind:

Page | 1
1. A well-designed questionnaire should meet the research objectives. This may seem obvious, but
many research surveys omit important aspects due to inadequate preparatory work, and do not
adequately probe particular issues due to poor understanding. To a certain degree some of this is
inevitable. Every survey is bound to leave some questions unanswered and provide a need for further
research but the objective of good questionnaire design is to 'minimise' these problems.

2. It should obtain the most complete and accurate information possible. The questionnaire designer
needs to ensure that respondents fully understand the questions and are not likely to refuse to
answer, lie to the interviewer or try to conceal their attitudes. A good questionnaire is organised and
worded to encourage respondents to provide accurate, unbiased and complete information.

3. A well-designed questionnaire should make it easy for respondents to give the necessary
information and for the interviewer to record the answer, and it should be arranged so that sound
analysis and interpretation are possible.

4. It would keep the interview brief and to the point and be so arranged that the respondent(s) remain
interested throughout the interview.

Each of these points will be further discussed throughout the following sections. Figure 4.1 shows how
questionnaire design fits into the overall process of research design that was described in chapter 1 of
this textbook. It emphasises that writing of the questionnaire proper should not begin before an
exploratory research phase has been completed.

Figure 4.1 The steps preceding questionnaire design

Even after the exploratory phase, two key steps remain to be completed before the task of designing
the questionnaire should commence. The first of these is to articulate the questions that research is
intended to address. The second step is to determine the hypotheses around which the questionnaire
is to be designed.

It is possible for the piloting exercise to be used to make necessary adjustments to administrative
aspects of the study. This would include, for example, an assessment of the length of time an
interview actually takes, in comparison to the planned length of the interview; or, in the same way, the
time needed to complete questionnaires. Moreover, checks can be made on the appropriateness of
the timing of the study in relation to contemporary events such as avoiding farm visits during busy
harvesting periods.

Preliminary decisions in questionnaire design

There are nine steps involved in the development of a questionnaire:

1. Decide the information required.

2. Define the target respondents.
3. Choose the method(s) of reaching your target respondents.
4. Decide on question content.
5. Develop the question wording.
6. Put questions into a meaningful order and format.
7. Check the length of the questionnaire.
8. Pre-test the questionnaire.
9. Develop the final survey form.

Deciding on the information required

Page | 2
It should be noted that one does not start by writing questions. The first step is to decide 'what are the
things one needs to know from the respondent in order to meet the survey's objectives?' These, as
has been indicated in the opening chapter of this textbook, should appear in the research brief and
the research proposal.

One may already have an idea about the kind of information to be collected, but additional help can
be obtained from secondary data, previous rapid rural appraisals and exploratory research. In respect
of secondary data, the researcher should be aware of what work has been done on the same or
similar problems in the past, what factors have not yet been examined, and how the present survey
questionnaire can build on what has already been discovered. Further, a small number of preliminary
informal interviews with target respondents will give a glimpse of reality that may help clarify ideas
about what information is required.

Define the target respondents

At the outset, the researcher must define the population about which he/she wishes to generalise from
the sample data to be collected. For example, in marketing research, researchers often have to
decide whether they should cover only existing users of the generic product type or whether to also
include non-users. Secondly, researchers have to draw up a sampling frame. Thirdly, in designing the
questionnaire we must take into account factors such as the age, education, etc. of the target
respondents.

Choose the method(s) of reaching target respondents

It may seem strange to be suggesting that the method of reaching the intended respondents should
constitute part of the questionnaire design process. However, a moment's reflection is sufficient to
conclude that the method of contact will influence not only the questions the researcher is able to ask
but the phrasing of those questions. The main methods available in survey research are:

· personal interviews
· group or focus interviews
· mailed questionnaires
· telephone interviews.

Within this region the first two mentioned are used much more extensively than the second pair.
However, each has its advantages and disadvantages. A general rule is that the more sensitive or
personal the information, the more personal the form of data collection should be.

Decide on question content

Researchers must always be prepared to ask, "Is this question really needed?" The temptation to
include questions without critically evaluating their contribution towards the achievement of the
research objectives, as they are specified in the research proposal, is surprisingly strong. No question
should be included unless the data it gives rise to is directly of use in testing one or more of the
hypotheses established during the research design.

There are only two occasions when seemingly "redundant" questions might be included:

· Opening questions that are easy to answer and which are not perceived as being "threatening",
and/or are perceived as being interesting, can greatly assist in gaining the respondent's involvement
in the survey and help to establish a rapport.

This, however, should not be an approach that should be overly used. It is almost always the case
that questions which are of use in testing hypotheses can also serve the same functions.

Page | 3
· "Dummy" questions can disguise the purpose of the survey and/or the sponsorship of a study. For
example, if a manufacturer wanted to find out whether its distributors were giving the consumers or
end-users of its products a reasonable level of service, the researcher would want to disguise the fact
that the distributors' service level was being investigated. If he/she did not, then rumours would
abound that there was something wrong with the distributor.

Develop the question wording

Survey questions can be classified into three forms, i.e. closed, open-ended and open response-
option questions. So far only the first of these, i.e. closed questions has been discussed. This type of
questioning has a number of important advantages;

· It provides the respondent with an easy method of indicating his answer - he does not have to think
about how to articulate his answer.

· It 'prompts' the respondent so that the respondent has to rely less on memory in answering a
question.

· Responses can be easily classified, making analysis very straightforward.

· It permits the respondent to specify the answer categories most suitable for their purposes.

Disadvantages are also present when using such questions

· They do not allow the respondent the opportunity to give a different response to those suggested.

· They 'suggest' answers that respondents may not have considered before.

With open-ended questions the respondent is asked to give a reply to a question in his/her own
words. No answers are suggested.

Example: "What do you like most about this implement?"

Open-ended questions have a number of advantages when utilised in a questionnaire:

o They allow the respondent to answer in his own words, with no influence by any specific
alternatives suggested by the interviewer.

o They often reveal the issues which are most important to the respondent, and this may reveal
findings which were not originally anticipated when the survey was initiated.

o Respondents can 'qualify' their answers or emphasise the strength of their opinions.

However, open-ended questions also have inherent problems which means they must be treated with
considerable caution. For example:

o Respondents may find it difficult to 'articulate' their responses i.e. to properly and fully explain
their attitudes or motivations.

o Respondents may not give a full answer simply because they may forget to mention important
points. Some respondents need prompting or reminding of the types of answer they could give.

o Data collected is in the form of verbatim comments - it has to be coded and reduced to
manageable categories. This can be time consuming for analysis and there are numerous
opportunities for error in recording and interpreting the answers given on the part of
interviewers.

Page | 4
 o Respondents will tend to answer open questions in different 'dimensions'. For example, the
question: "When did you purchase your tractor?", could elicit one of several responses, viz:

"A short while ago".

"Last year".
"When I sold my last tractor".
"When I bought the farm".

Such responses need to be probed further unless the researcher is to be confronted with responses
that cannot be aggregated or compared.

It has been suggested that the open response-option questions largely eliminate the disadvantages of
both the afore-mentioned types of question. An open response-option is a form of question which is
both open-ended and includes specific response-options as well. For example,

What features of this implement do you like?

· Performance
· Quality
· Price
· Weight
· Others mentioned:

The advantages of this type of question are twofold:

· The researcher can avoid the potential problems of poor memory or poor articulation by then
subsequently being able to prompt the respondent into considering particular response options.

· Recording during interview is relatively straightforward.

The one disadvantage of this form of question is that it requires the researcher to have a good prior
knowledge of the subject in order to generate realistic/likely response options before printing the
questionnaire. However, if this understanding is achieved the data collection and analysis process can
be significantly eased.

Clearly there are going to be situations in which a questionnaire will need to incorporate all three
forms of question, because some forms are more appropriate for seeking particular forms of
response. In instances where it is felt the respondent needs assistance to articulate answers or
provide answers on a preferred dimension determined by the researcher, then closed questions
should be used. Open-ended questions should be used where there are likely to be a very large
number of possible different responses (e.g. farm size), where one is seeking a response described in
the respondent's own words, and when one is unsure about the possible answer options. The mixed
type of question would be advantageous in most instances where most potential response-options are
known; where unprompted and prompted responses are valuable, and where the survey needs to
allow for unanticipated responses.

There are a series of questions that should be posed as the researchers develop the survey
questions themselves:

"Is this question sufficient to generate the required information?"

For example, asking the question "Which product do you prefer?" in a taste panel exercise will reveal
nothing about the attribute(s) the product was judged upon. Nor will this question reveal the degree of
preference. In such cases a series of questions would be more appropriate.

"Can the respondent answer the question correctly?"

Page | 5
· An inability to answer a question arises from three sources:

· Having never been exposed to the answer, e.g. "How much does your husband earn?"

· Forgetting, e.g. What price did you pay when you last bought maize meal?"

· An inability to articulate the answer: e.g. "What improvements would you want to see in food
preparation equipment?"

"Are there any external events that might bias response to the question?"

For example, judging the popularity of beef products shortly after a foot and mouth epidemic is likely
to have an effect on the responses.

"Do the words have the same meaning to all respondents?"

For example, "How many members are there in your family?"

There is room for ambiguity in such a question since it is open to interpretation as to whether one is
speaking of the immediate or extended family.

"Are any of the words or phrases loaded or leading in any way?"

For example," What did you dislike about the product you have just tried?"

The respondent is not given the opportunity to indicate that there was nothing he/she disliked about
the product. A less biased approach would have been to ask a preliminary question along the lines of,
"Did you dislike any aspect of the product you have just tried?", and allow him/her to answer yes or
no.

"Are there any implied alternatives within the question?"

The presence or absence of an explicitly stated alternative can have dramatic effects on responses.
For example, consider the following two forms of a question asked of a 'Pasta-in-a-Jar' concept test:

1. " Would you buy pasta-in-a-jar if it were locally available?"

2. "If pasta-in-a-jar and the cellophane pack you currently use were both available locally, would you:

· Buy only the cellophane packed pasta?

· Buy only the pasta-in-a-jar product?
· Buy both products?"

The explicit alternatives provide a context for interpreting the true reactions to the new product idea. If
the first version of the question is used, the researcher is almost certain to obtain a larger number of
positive responses than if the second form is applied.

"Will the question be understood by the type of individual to be interviewed?"

It is good practice to keep questions as simple as possible. Researchers must be sensitive to the fact
that some of the people he/she will be interviewing do not have a high level of education. Sometimes
he/she will have no idea how well or badly educated the respondents are until he/she gets into the
field. In the same way, researchers should strive to avoid long questions. The fewer words in a
question the better. Respondents' memories are limited and absorbing the meaning of long sentences
can be difficult: in listening to something they may not have much interest in, the respondents' minds
are likely to wander, they may hear certain words but not others, or they may remember some parts of
what is said but not all.

Page | 6
"Is there any ambiguity in my questions?"

The careless design of questions can result in the inclusion of two items in one question. For
example: "Do you like the speed and reliability of your tractor?"

Page | 7