TEHNOMUS - New Technologies and Products in Machine Manufacturing Technologies

QUESTIONNAIRE CHARACTERISTICS
FOR EVALUATION OF MANAGEMENT
IN THE SMALL AND MEDIUM SIZE ENTERPRISES

Mirela NASTASIEA1, Costel MIRONEASA2

1
Faculty of Mechanical Engineering, Mechatronics and Management,
“Ştefan cel Mare” University of Suceava - [email protected]
2
Faculty of Mechanical Engineering, Mechatronics and Management,
“Ştefan cel Mare” University of Suceava - [email protected]

Abstract: Questionnaires are used in management, economics, business, psychology, public

opinion research and many other areas to collect data from human respondents. A well-
designed questionnaire can gather information about the overall performance of a product or
system, as well as information on specific components. In the past, the term questionnaire
almost invariably referred to a paper form, but now many surveys are administered by
computer assisted personal interview devices, telephone or web-based electronic forms. Surveys
are a traditional method of gathering information at a minimum cost to the researcher.

Keywords: questionnaire, management, small and medium size enterprises, survey,

questions, responses

range of information from a large number of

1. Introduction individuals, often referred to as respondents
A questionnaire is defined by Kirakowski Figure 1. Designing and developing
[Kirakowski, 2000] as a “method for the questionnaires involves two main components:
elicitation, the recording and the collecting of - one – deciding what the questionnaire needs
information”: to measure and
- method implies that a questionnaire is a tool - two – designing and testing questionnaire
rather than an end in itself; items to be good measures of the construct
- elicitation is the bringing out of information or trait under study [Fowler, 2002].
from respondents through questioning; The questions should convey the same
- the responses at questionnaire are recorded in meaning and function equally well across
various ways – video, voice, written text; respondents.
- collecting implies that by administering Respondents should understand all of the
questionnaires to many respondents, information presented in questions, which is
evaluators of them usually expect a only possible if the question is relevant, clear
compilation of the questionnaires (for in meaning and presents a manageable amount
achieve higher validity in the outcome of of material.
the questionnaires, higher numbers of Respondents should have the ability and
collections are recommended). motivation to answer. The questions should
When properly constructed and allow respondents to accurately express their
responsibly administered, questionnaires responses in a manner that reflects their actual
become a vital instrument of collecting a wide experience, attitudes, etc.

170
 TEHNOMUS - New Technologies and Products in Machine Manufacturing Technologies

Schuman and Kalton [Schuman & Kalton, 1985] said: “asking questions is a remarkably
efficient way to obtain information from and about people. Whether the information desired is
factual or consist of expressions of attitude, beliefs, judgements or whatever, one can obtain an
enormous amount of it a relatively brief time provided only that the person answering is able
and willing to respond.”

Payne [Payne, 1951] stated that asking the right

questions that provide valid and reliable information for
making decisions as well as testing a theory is probably
as much an art as the other aspects of research.

Kinnear and Taylor [Kinnear & Taylor, 1996] :

“Questionnaire design is more an art form than a
scientific undertaking. No steps, principles, or
guidelines can guarantee an effective and efficient
questionnaire.”
Peterson [Peterson, 2000] claimed that the data gathered from questionnaires might be useless
if a researcher does not ask the right question in the right way even though the other aspects of
the research is well designed; even worse, the wrong data might lead the researcher in a wrong
direction.
Figure 1: Opinions about questionnaire

A goal is well-defined when the following analysis. It is mainly made up of a list of

three criteria are met: questions, but should also include clear
a) a goal can be stated as an outcome; instructions and space for answers or
b) the outcome can be specified as actions; administrative details.
c) the outcome can be numerically measured. Questionnaires are familiar to most people
Once the objective of the study is defined and they translate the research objective into
and specified clearly, the research effort is specific questions and it needs to be clear from
concentrated on collecting data. That involves the outset how the findings will be used. The
lengthy and costly processes such as: answers to those questions provide the data for
a) designing and pretesting the testing the research hypothesis; questions must
questionnaires; also interest the respondents enough that they
b) identifying the respondents; will provide the information. One step in
c) training of field interviewing staffs; designing questionnaire is to create a
d) checking data to ensure accuracy of data conceptual model: this includes specifying the
gathered and collected; research problem, the purpose of the research,
e) coding, punching and tabulating the data the research design, the variables and
collected. hypothesis and operational definitions and
valid and reliable measures of the variables, as
2. Questionnaire characteristics well as the intended population, and the plans
for data analysis. To produce the questionnaire
A questionnaire is simply a tool for writing the introduction, the statement of
collecting and recording information about a informed consent, and the questions and
particular issue of interest; Babbie [Babbie, responses, as well as designing the overall
1990] said that a questionnaire is a document format of the questionnaire. Brace [Brace,
containing questions and other types of items 2008] said that only relevant questions to the
designed to solicit information appropriate to subject should be asked of participants. The

171
 TEHNOMUS - New Technologies and Products in Machine Manufacturing Technologies

researcher’s duty is to construct problems and computational problems [Conrad

understandable and objective questions for & Blair, 1996]. Potential errors can occur if
respondents because using difficult language, the respondent interprets the question
ambiguous or influential quantifiers’ results in differently from the way the researcher
getting wrong or no responses from the intended. Error can also occur when different
participant. Ambiguous questions contain respondents interpret the question differently
unfamiliar words and/or words that have from each other. Pre-test (to field-test it by
multiple meanings. Factors in interpretation of performing a pilot study in a small
questions that affect respondent’s ability to representative sample of respondents from the
understand a questionnaire was categorized by target group) the questionnaire is a mod to
Conrad and Blair into five types: lexical revise, and to conduct a pilot test of how the
(terminology) problems, inclusion/exclusion questionnaire will be used.
problems, temporal problems, logical

1 Review information requirements of the problem, opportunity or decision to be made

Develop and prioritize a list of potential research questions o provide the

2
required information
Evaluate each potential research question to see if participants can
3
understand, can answer the questions

4 Determine types (open-end or/and closed-end) of questions to be asked

Decided on specific wording of each question to be asked

5

Determine questionnaire structure

6

7 Evaluate questionnaire

Feedback
Figure 2: Steps in questionnaire design

Peterson [Peterson, 2006] suggested a As another way of classifying types of

seven step-framework for constructing questions, Czaja and Blair [Czaja & Blair,
questionnaire, Figure 2 and explained “unless 1996] summarized three types of
a researcher understands the information questionnaires: factual questionnaires
requirements – what information is needed (questionnaires usually ask about public or
and how that information will be used – no observable information), opinion
attempt should be made to construct a questionnaires (questionnaires ask what
questionnaire. The effort would be a waste of respondents think about something which can
time for the researcher and the results would interpret as opinion) and attitude
be of little value.” There are several ways to questionnaires (try to gather respondents’
classify questionnaire such as closed – ended internal response to events, situations or usage
or open – ended [Brace, 2008], [Chelcea, of products).
2001], Figure 3.

172
 TEHNOMUS - New Technologies and Products in Machine Manufacturing Technologies

QUESTIONNAIRES
open – ended questionnaire
close – ended questionnaire structured questionnaire
unstructured questionnaire mixed questionnaire
pictorial questionnaire postal questionnaire
telephone questionnaire electronic questionnaire
factual questionnaire personally administered question.
opinion questionnaire attitude questionnaire
QUESTIONNAIRE

QUESTIONS RESPONSES
closed questions refusal
open questions distortion
factual questions stalling
convergent questions a lie
divergent questions out of context
evaluative questions a direct response
combination a honest response
dichotomous questions partially answering
filter or contingency questions avoiding the answer
questions based on level of measurement

Figure 3:. Questionnaire as a diamond

(questions like pavilion, responses like crown and questionnaire like diamond)

A questionnaire is considered to have Over the years, the practice of quality

content validity if the included items are management has been identified and
representative of the content area in which the awareness has been created as one of
investigator is interested, there are two key organization’s most important key ingredient
facets to content validity: content relevance for success and global competitiveness. A
and content coverage [Nunnaly, 1978], small and medium enterprise that implements
[Streiner & Norman, 2008]: quality management practices may have a
- content relevance refers to the extent to competitive advantage over its competitor.
which each item is related to the content Through questionnaires can investigate the
area under investigation (items that fail to impact on the implementation of quality
show adequate content relevance may management practices on the performance and
introduce measurement error and growth of SMEs, can determine the
discriminate among respondents on some relationship between quality management and
dimension irrelevant to what is being performance mostly in the developed world
assessed); and can study evaluating management in
- content coverage on the other hand, reflects SMEs, Figure 4.
the extent to which separate domains within Quality has been defined by Karapetrovic
the content area of interest are represented and Willborn [Karapetrovic & Willborn, 1997]
by one or more items (if not, respondents as “the ability of a product to satisfy stated or
may differ in some important aspects, but implied requirements”. Total Quality
this difference might not be reflected in the Management is seen as a tool to improve
final score obtained from the questionnaire organizational performance in both large and
or its individual subscales). small and medium organizations and in any
part of the world. Wessel and Burcher [Wessel

173
 TEHNOMUS - New Technologies and Products in Machine Manufacturing Technologies

& Burcher, 2004] said that “quality and micro operations towards the needs of
management in general deals with internal and external customers”. The
permanently redirecting a company’s macro dependent variable is performance.

Literature review
(management,
Final
SMEs, quality, Pilot questioner questionnaire
evaluation,
performance

Administering
questionnaire
Creating data base Conclusions
(contact people, Data analysis
(data collection) (information)
web, e-mail)

Figure 4: Steps in questionnaire administration

For a product, the technological employee empowerment, quality culture,

sophistication is one of the essential aspects of employee satisfaction, social responsibility,
its performance. A questionnaire can be used how new products could address latent needs.
to obtain businesses’subjective view of the How SMEs approach implementing
quality management and the quality of the quality management varies, depending on the
products. The questionnaire can be divided industry, the level of maturity of their product,
into two parts: the first focuses on the common their strategy and objectives, organizational
characteristics of the SME, by general culture and a lot of other factors.
questions related to monitoring quality and its Criteria of questionnaire in management
relation to the SME’competitiveness and the quality is presented in Figure 5.
second part of questionnaire ascertains Each type of questionnaire has its own
qualitative characteristics to analyze quality, strengths and weaknesses. First, it is important
including quality of business management and to ask questions that are clear and specific,
evaluation of performance management. without unwarranted assumptions or imply a
Feedback from the pilot questionnaire can be desired answer, and that each respondent will
used to clarify whether the questions were be able to answer. Questions that use simple
easy to understand, whether there were any and concrete language are more easily
other questions that needed to be included. understood by respondents:
Conclusions of the questionnaires may - if a question is open-ended, it should be
have information about key performance evident to respondents that they can answer
indicators: continuous improvement and in their own words;
innovation, information and performance - if questions are closed – ended should
measurement, process management, strategic include all reasonable responses [Bulmer,
planning, process control, product and service 2004]
design, flexibility, quality systems, people When similar questions are administered
training, top management commitment, simultaneously to a large number of people the
teamwork, employee involvement, supplier acquired data are more identical, correct and
management, communication, rewards and standard.
recognition, human resource management,

174
 TEHNOMUS - New Technologies and Products in Machine Manufacturing Technologies

Effectiveness Learnability

Criteria of
Flexibility questionnaire in Attitude
management
quality
Understandability Operability

Errors
Efficiency Attractiveness
Figure 5: Crystal questionnaire

Blaxter et al. [Blaxter et al, 2006] divide rates vary widely from one questionnaire to
questionnaires into “seven basic question types: another 10% - 90%. Respondents may not be
quantity or information, category, list or motivated enough to complete a whole
multiple choice, scale, ranking, complex grid questionnaire and the web questionnaire is
or table, and open-ended.” therefore the only tool that the researcher has
Particular attention should be paid to how available and its design is very important for
questions are ordered in the questionnaire [Yin, obtaining survey data of the desired quality.)
2009], [Sarantakos, 2007]. The order in which [Silverman, 2010],[Sushil & Stohr, 2014].
questions are asked can influence how people Over the past decades, performance
respond: management is defined as the process where
- if closed-ended questions that relate to the steering of the organization takes place
topic are placed before open-ended through the systematic definition of mission,
questions, respondents are much more strategy and objectives of the organization,
likely to mention concepts or considerations making these measurable through critical
raised in those earlier questions when success factors and key performance indicators,
responding to the open-ended questions; in order to be able to take corrective actions to
- order effects for closed-ended opinion keep the organization on track [Flick, 2006].
questions are contrast effects (the order Evaluating through questionnaires and
results in greater differences in responses) comparing performance within and between
and assimilation effects (responses are more SMEs, the examinations of multiple quality
similar as a result of their order). inputs and multiple performance outputs can
Assimilation effects occur when responses be quite difficult when there are multiple,
to two questions are more consistent or inter-related dimensions to performance.
closer together because of their placement The results of questionnaire may include
in the questionnaire. data about best quality management practices
One disadvantage of questionnaires is the to improved performance, specifically quality
possibility of low response rates (response and innovation performance.

175
 TEHNOMUS - New Technologies and Products in Machine Manufacturing Technologies

3. Conclusions qualitative, qualitative with some statistics and

purely qualitative”.
Research efforts can be classified into
Today, books provide limited information
three broad areas:
on questionnaires as a data collection method,
1 - methods to increase response rate,
there are only a few texts that deal specifically
thereby reducing nonresponse bias;
with quantitative methods: [Oakshott, 2009],
2 - methods to increase response quality
[Swift & Piff, 2010], [Easterby-Smith et al.
which would, in turn, reduce response bias;
2012], [Saunders et al. 2012].
3 - methods of assessing the degree and
Through questionnaires, researchers can
direction of bias with attempts to correct
be able to assess correlations between SME
existing bias yielding closer estimates of the
performance and the extent to which total
population values.
quality management was implemented in the
Questionnaire are doubtless source of
organization, relationships between total
obtaining data in any research and the
quality management and general
researchers should ensure that is “valid,
organizational performance including quality,
reliable and unambiguous” [Flick, 2006]. The
to identify best quality management practice
researcher must be careful at the quality of the
via simultaneous consideration of multiple
questions of the questionnaire.
Since the beginning of 21st century, the inputs and multiple outputs.
boom of information technology leads a lot of
social affairs to paperless. Nowadays, because References
lots of paper would be used for printing and a 1. [Babbie, 1990] Babbie, E., Survey
lot of time is spent for distributing, paperless Research Methods, Second Edition,
questionnaire through the Internet is gradually California: Wadsworth Publishing
emphasized and adopted by researchers. Company, 1990
Considering that the design of questions 2. [Blaxter et al., 2006] Blaxter, L., Hughes,
and questionnaires is an art as well as a C. and Tight, M., How to research,
science, researchers who compose Berkshire: Open University Press, 2006
questionnaires should review questions from 3. [Brace, 2008] Brace, I., Questionnaire
earlier surveys before writing their own. It is design: how to plan, structure and write
important for researchers to reflect on their survey material for effective market
limitations and the consequent limitations of research, London, Philadelphia: Kogan
their own research. Questionnaires used in Page, 2008
survey-based research are often arranged in 4. [Brown, 2001] Brown, J.D. Using Survey
sections that contains item that are closely in Language Programs, Cambridge
interrelated, serving one or more themes. The University Press, 2001
resulting dataset may have a large number of 5. [Bulmer, 2004] Bulmer,
variables, which poses special analytic M., Questionnaires, 1 st
edition, Sage
challenges for dealing with missing values. Benchmarks in Social Science Research
Selinger and Shohamy [Selinger & Methods, edited by: Bulmer, M., Sage
Shohamy, 1989] are of the opinion that closes- Publications, London, 2004
ended questionnaires are more efficient 6. [Chelcea, 2001] Chelcea, S., Metodologia
because of their ease of analysis, and in 2000 cercetării sociologice. Metode cantitative
Gillham to admits the difficulty of analyzing şi calitative, Editura Economică, Bucureşti,
open-ended questionnaires but argues that 2001
“open questions can lead to a greater level of 7. [Conrad & Blair, 1996] Conrad F., Blair J. ,
discovery.” [Gillham, 2000]. Brown [Brown, Proceedings of the Survey Research
2001] divides the research design into four Methods Section of the American
parts: “purely statistical, statistical with some Statistical Association, American
Statistical Association, Alexandrai, 1996

176
 TEHNOMUS - New Technologies and Products in Machine Manufacturing Technologies

8. [Czaja & Blair, 1996] Czaja, R. and Blair, 3rd edition, New York: Random House,
J., Designing surveys: a guide to decisions 1985
and procedures, Thousand Oaks, CA: Pine 23. [Selinger & Shohamy, 1989] Selinger,
Forge Press, 1996 H.W. and Shohamy, E., Second language
9. [Easterby-Smith, 2012] Easterby-Smith, research methods, Oxford: OUP, 1989
M., Thorpe, R., Jackson, P. and Lowe, A., 24. [Silverman, 2010] Silverman, D., Doing
Management Research, 4th edition, Sage, Qualitative Research: A Practical
London, 2012 Handbook, 3rd ed., Sage, London, 2010
10. [Flick, 2006] Flick, U., An introduction to 25. [Streiner & Norman, 2008] Streiner, D.L.
qualitative research, London, Sage, 2006 and Norman, G.R., Health measurement
11. [Fowler, 2002] Fowler, F.J., Survey scales: A practical guide to their
research methods, Third edition, Thousand development and use (third edition),
Oaks: Sage, 2002 Oxford: Oxford University Press, 2008
12. [Gillham, 2000] Gillham, B., Developing a 26. [Sushil & Stohr, 2012] Sushil E. A., and
questionnaire, London: Continuum, 2000 Stohr, The Flexible Enterprise, Department
13. [Karapetrovic & Willborn, 1997] of Management Studies, Indian Institute of
Karapetrovic, S. and Willborn, W., Technology Delhi, Hauz Khas, New Delhi,
Creating zero-defect students, TQM India, 2014
Magazine, 1997 27. [Swift & Piff, 2010] Swift, L. and Piff, S.
14. [Kinnear & Taylor, 1996] Kinnear, T.C. Quantitative Methods for Business,
and Taylor, J.R., Marketing research: an Management and Finance, 3rd ed.,
applied approach, New York: McGraw- Palgrave, Basingstoke, 2010
Hill, 1996 28. [Wessel & Burcher, 2004] Wessel, R. and
15. [Kirakowski, 2009] Kirakowski, J., Ireland Burcher P., Six sigma for small and
http://www.ucc.ie/ hfrg/ medium sized enterprises, TQM Magazine,
resources/qfaq1.html, , 2000 Bedford, 2004
16. [Nunnaly, 1978] Nunnaly, J.C., 29. [Yin, 2009] Yin, R.K., Case study
Psychometric theory (second edition), New research: Design and methods, 4th edition,
York: McGraw-Hill, 1978 Los Angeles: Sage Publications, 2009
17. [Oakshott, 2009] Oakshott, L., Essential
Quantitative Methods: For Business,
Management and Finance, 4th edition,
Palgrave, Basingstoke, 2009
18. [Payne, 1951] Payne, S., The art of asking
questions, Princeton, Princeton University
Press, 1951
19. [Peterson, 2000] Peterson, R.A.,
Constructing effective questionnaires,
Thousand Oaks: Sage Publications, 2000
20. [Sarantakos, 2007] Sarantakos, S., A
toolkit for quantitative data analysis,
Palgrave MacMillan, Hampshire, 2007
21. [Saunders, 2012] Saunders, M.N.K.,
Thornhill, A. and Lewis, P., Research
Methods for Business Students, 6th ed., FT
Pretince-Hall, London, 2012
22. [Schuman & Kalton, 1985] Schuman, H.
and Kalton, G., Survey Methods – The
Handbook of Social Psychology, volume 1,

177