INT J LANG COMMUN DISORD, MAY–JUNE 2011,

VOL. 46, NO. 3, 251–260

Paradigms, pragmatism and possibilities: mixed-methods research in speech

and language therapy
Margaret Glogowska
Faculty of Health and Life Sciences, University of the West of England, Bristol, UK

(Received 8 October 2008; accepted 6 July 2010)

Abstract
Background: After the decades of the so-called ‘paradigm wars’ in social science research methodology and the
controversy about the relative place and value of quantitative and qualitative research methodologies, ‘paradigm
peace’ appears to have now been declared. This has come about as many researchers have begun to take a ‘pragmatic’
approach in the selection of research methodology, choosing the methodology best suited to answering the research
question rather than conforming to a methodological orthodoxy. With the differences in the philosophical
underpinnings of the two traditions set to one side, an increasing awareness, and valuing, of the ‘mixed-methods’
approach to research is now present in the fields of social, educational and health research.
Aims: To explore what is meant by mixed-methods research and the ways in which quantitative and qualitative
methodologies and methods can be combined and integrated, particularly in the broad field of health services
research and the narrower one of speech and language therapy.
Main contribution: The paper discusses the ways in which methodological approaches have already been combined
and integrated in health services research and speech and language therapy, highlighting the suitability of mixed-
methods research for answering the typically multifaceted questions arising from the provision of complex interven-
tions. The challenges of combining and integrating quantitative and qualitative methods and the barriers to the
adoption of mixed-methods approaches are also considered.
Conclusions & Implications: The questions about healthcare, as it is being provided in the 21st century, calls for
a range of methodological approaches. This is particularly the case for human communication and its disorders,
where mixed-methods research offers a wealth of possibilities. In turn, speech and language therapy research should
be able to contribute substantively to the future development of mixed-methods research.

Keywords: research, mixed methods, speech and language therapy.

What this paper adds

Debate over the place and value of quantitative and qualitative research methodologies in social science research
has a long history – the so-called ‘paradigm wars’. However, taking a more pragmatic stance, researchers have
begun to base methodological choice on the need to answer a research question rather than on philosophical
alignment. As a result, a mixed-methods approach to research has emerged in a number of disciplines, including
health. This paper provides an overview of what is meant by mixed-methods research. It describes ways in which
different methodological approaches have been combined and integrated in health services research generally and
in speech and language therapy in particular. The challenges of conducting mixed-methods research and barriers to
its implementation are discussed. The potential for the use of this approach in speech and language therapy research
is also explored.

Paradigm war and peace stressed. This idea of incompatibility was predicated
on the supposedly fundamentally opposing ‘worldviews’
For decades in the social sciences, the so-called ‘paradigm
of quantitative and qualitative research, regarding the
wars’ raged over the worth and position of quantitative
nature of reality and whether it exists ‘out there’
and qualitative research (Bryman 1988, 2006a, Oakley
(ontology) and how it is possible to ‘know’ about it
2000). In this long-running debate, the incompat-
(epistemology). Quantitative research has been linked
ibility of quantitative and qualitative research was

Address correspondence to: Margaret Glogowska, Faculty of Health and Life Sciences, University of the West of England, Glenside Campus,
Blackberry Hill, Bristol BS16 1DD, UK; e-mail: [email protected]
International Journal of Language & Communication Disorders
ISSN 1368-2822 print/ISSN 1460-6984 online  c 2010 Royal College of Speech & Language Therapists
DOI: 10.3109/13682822.2010.507614
252 Margaret Glogowska
with a ‘positivist’ worldview, and qualitative with an that it can be ‘scientific’, while some qualitative
‘interpretivist’ one. researchers may be unprepared to take a more relaxed
With regard to ontology, quantitative research tends stance on reality. However, influential authors are now
to view the existence of an independent reality as declaring that the paradigm war is over (Patton 1988,
unproblematic and therefore capable of being studied, Barbour 1999, Bryman 2006a, Morgan 2007). They
although acknowledging that it can only be approached acknowledge that the forerunners of this movement
imperfectly and partially. For some prominent qualita- for paradigm peace have tended to be researchers
tive researchers, reality is more a construction shaped in in applied fields, for example evaluation research,
people’s minds with no independent existence (Murphy who have shunned methodological purity in favour
and Dingwall 1998). Thus, different minds may of pragmatism, advocating a ‘paradigm of choices’
perceive different psychologically, socially politically (Patton 1988, 2002). Putting to one side the claims
and historically conditioned ‘realities’, meaning that and counter-claims of the underpinning philosophies,
multiple, and possibly contradictory, realities will exist the pragmatists have argued that the dichotomy
alongside each other. between quantitative and qualitative research is a false
In terms of epistemology, quantitative research takes one and have proceeded to loosen the connection
as read the independence of phenomena and the between methods and methodological stances. They
possibility of the researcher maintaining an objective have demonstrated that, in order to answer research
stance towards the researched, creating the idea of ‘value- questions usefully and efficiently, recourse to both
free’ investigation. In qualitative research a more interac- methodologies is necessary. This is the ‘horses for
tive relationship between researcher and researched is courses’ (Murphy et al. 1998) or ‘toolkit’ approach
posited. The qualitative researcher cannot be objective (Ritchie and Lewis 2003), whereby methodology is
and cannot, therefore, produce an, in any way, privileged chosen for its aptness for answering the research question
overview or objective account, because of their pre- posed rather than because of any pre-existing philosoph-
existing assumptions, attitudes and beliefs. These cannot ical commitment. Such pragmatism is a significant
be set aside but have to be acknowledged and handled development. If the ‘incommensurability’ (Kuhn 1970)
in the conduct of the research (Ritchie and Lewis 2003). of quantitative and qualitative paradigms is substan-
Quantitative and qualitative approaches have been tially rejected (Morgan 2007), then new possibilities for
further polarized as deductive and inductive respectively. combining and integrating quantitative and qualitative
Put simply, deduction refers to the testing out of prior approaches beckon.
theories and hypotheses through a logical process of
evidence-gathering. Induction, on the other hand, relies
on the gathering of evidence through observations of Paradigms and speech and language therapy
the world to generate theories and hypotheses. The To some extent during this time, speech and
main differences between the quantitative and qualita- language therapy (SLT) has been engaged in its own
tive paradigms are summarized in table 1. version of ‘paradigm wars’. The so-called ‘troublesome’
Positions remain entrenched for some—some (though largely rather misinterpreted and misrepre-
quantitative researchers may believe that qualitative sented) randomized controlled trials (Pring 2004) have
research is ‘anecdotal’ and therefore remain unconvinced engendered unusual publicity—some good but most

Table 1. Differences between the quantitative (positivist) and qualitative (interpretivist) paradigms

Positivist (quantitative) Interpretivist (qualitative)

Image of reality Tangible/amenable to investigation/ Socially constructed/multiple ‘realities’/needs to be
capable of being subdivided into component considered holistically/context very important
parts/can be divorced from context
Aims To test out hypotheses/to generalize to wider To generate hypotheses/to provide detailed
populations descriptive accounts of phenomena
Methods used Counting and controlled measurement; Participant observation; non-participant
experiments/quasi-experiments; surveys; numerical observation; in-depth interviewing; focus groups;
records; structured observations documentary analysis
Relationship of Distant/independent—the researcher stands Close/interactive—the researcher stands ‘inside’ the
researcher and subject ‘outside’ the object, process, etc. research process
Setting for the research ‘Laboratory’/more artificial setting ‘Nature’/less artificial setting
Data ‘Hard’, reliable, replicable ‘Rich’, deep, valid
Generalizability Generalizability across time and across population Time and context are crucial in determining
is possible whether findings apply elsewhere
Mixed-methods research 253
of it bad (Meikle et al. 1979, David et al. 1982, ‘multi-strategy’ and ‘mixed methodology’ research. This
Lincoln et al. 1984—examples of evaluation of aphasia brave new world of ‘mixed-methods research’ has been
therapy; and Glogowska et al. 2000—an evaluation of increasingly taken up in social, educational and health
therapy with pre-school children). In the wake of this, fields (Johnstone 2004, Bryman 2006a, Moran Ellis et al.
some therapists have been persuaded that randomized 2006, O’Cathain et al. 2007a). The approach is finding
controlled trials (RCTs) were misapplied in SLT in international favour, with journals and handbooks solely
the first place and that the profession would be better concerned with mixed-methods research proliferating.
off without them. On the other hand, there have From this wide literature, a number of broad arguments
been attempts to reassure therapists about research justifying the use of mixed-methods research emerge
methodologies and to debunk some of the myths that (O’Cathain et al. 2007b):
have grown up, particularly around trial methodology
(Fitz-Gibbon 1986, Siegel and Young 1987, Glogowska • Building a comprehensive picture: where using
et al. 2001). However, some in the profession have a combination of quantitative and qualitative
been tempted metaphorically to place all their research methods can allow a phenomenon to be described
eggs in the qualitative basket, believing that qualita- and explained broadly and comprehensively.
tive methodologies apply more intuitively to the area • Increasing validity: checking the findings from one
of human communication and its disorders and that method alongside those of another.
the premise of co-construction of knowledge between • Developing: where one method is enhanced or
researcher and researched in qualitative research is facilitated by the use of another.
inherently more in keeping with the values of therapy. • Empowering: allowing normally marginalized
However, while there have been the rather groups to have a voice in research concerning
public wrangles about the ‘appropriateness’ of certain them.
methodologies in SLT, it is clear that a certain degree • Undertaking a mixed method study instead of a
of pragmatism has always prevailed here too. SLT is an single-method study which is impracticable.
applied field and largely speaking, it has worked within • Using one method to salvage another method
the broad understanding that some research designs are which is running into difficulty.
good at looking at some things, while other designs
are better at looking at others. At a grass roots level Mixed-methods research and health
in research, it has often been accepted that there are services research
aspects of human beings and their health, which call The use of mixed methods in health services research
for measurement and quantification—for example, the is becoming increasingly common (O’Cathain et al.
numbers of cases of a condition in the UK, the effects of 2007a). The motivation for this is likely to stem from the
treatments compared with other treatments, the costs of multidimensionality of factors influencing and affecting
setting up or maintaining clinical services. At the other health and illness and the multidisciplinary nature of
end of the spectrum, it is acknowledged that there are health services themselves (Ong 1993). Quite crucially,
aspects which call for a more qualitative approach— also, as Pope and Mays (1995) point out:
patients’ understanding of their diagnosis of a particu-
lar condition, their experiences of receiving particular because health care deals with people and people are on
treatments, their perceptions of how a service meets the whole more complex than the subject of the natural
their individual needs or not. Thus, paradigm peace sciences, there is a whole set of questions about human
may have been upon us in SLT for some time, as individ- interaction and how people interpret interaction which
ual clinicians, their service managers and policy makers health professionals may need answers to (p.43)
grapple with the plethora of research findings from Thus, alongside the issues of complexity, the concern
both the different traditions of research. As elsewhere, to introduce a more human focus, a more patient-
this de facto acceptance of both quantitative and centred approach to their studies has also appealed to
qualitative research is important. Consequently, new researchers. Indeed, the broad emancipatory justifica-
methodological possibilities of combining and integrat- tion for using mixed-methods research—empowering
ing these research approaches—the so-called mixed- less than powerful groups to find a voice—finds a signifi-
methods research—now open up for the profession. cant resonance in this field.

Mixed-methods research—what is it? Why and how is mixed-methods research

in health being undertaken?
‘Mixed-methods research’ is becoming the standard
terminology for research involving both quantitative A number of strategic reasons for adopting a mixed-
and qualitative methods, although it is still sometimes methods approach in health research are advanced
referred to as ‘multi(-)methods’, ‘multiple method’, and all, to some extent, take as their starting point
254 Margaret Glogowska
the deficiency of single-method approaches to tackle Barbour (1999) describes a number of ways in
research questions (Barbour 1999). They include the which qualitative research methods may contribute to
need for different methods at different stages in a study research taking place primarily within the quantita-
to fulfil separate functions, for example, using material tive paradigm. These include providing insights into
from qualitative interviews to inform the construction the pitfalls which may surround data collection and
of a survey questionnaire or attitude scale, which can how outcome measures may be subverted; identifying
help to enhance their accuracy and relevance. relevant and important factors in need of exploration
A reason also given for using mixed methods which were not previously salient to the researcher; being
is to provide ‘triangulation’, a term borrowed from able to provide explanations for surprising or discrepant
navigation, to denote the use of one part of a study quantitative findings and having a part in the generation
to act as a ‘check’ on another or of using more than of hypotheses and research questions.
one method to explore and explain a single concept. The purpose for which the methods are chosen
It was argued that comparing results from different will also determine the order in which they are used.
methods would enhance the confidence of the researcher Thus within a principally quantitative study qualitative
that the concept had been accurately measured— methods may be employed as a preliminary to it, for
the ‘increased validity’ model of triangulation—just example the use of individual interviews or focus groups
as multiple geographical readings might ensure more to inform an intervention study, or as part of a follow-up,
accurate mapping of a territory. The assumption was for example using qualitative methods after a survey to
made that one method cancelled the weaknesses of help understand why patient satisfaction may be higher
another, as it had different opportunities for error and in some areas than others.
bias. Therefore, where congruence in findings between Conversely, as Barbour (1999) asserts, within a
methods occurred, a greater confidence in the results principally qualitative study, quantitative methods can
could be claimed (Moran Ellis et al. 2006). assist with sorting and counting subjects and issues
However, as Morgan (1998) points out, triangu- emerging in large amounts of qualitative data which
lation has come to mean different things to different then make the data amenable to further thematic
people and considers it more appropriate to apply analysis. Quantitative methods can be used to inform the
the terms ‘convergence’ or ‘confirmation’ to the task sampling strategies qualitative researchers employ and
of comparing and checking findings across methods. the quantitative systematic review methodology has also
Moreover, he displays some scepticism about researchers’ influenced the development of the distinctive field of
interest in producing ‘convergent’ findings and the lack meta-ethnography, whereby qualitative studies’ findings
of time and resources to find ‘the same thing twice’ can be systematically retrieved, reviewed and synthesized
(p. 365). Instead he suggests that health researchers (Campbell et al. 2003a, Britten et al. 2002).
have pursued ‘complementarity’, opting to use the In practice, quantitative methods may provide the
strengths of one method to supplement another method, necessary preliminary information suggesting areas for
thus providing greater elaboration and understanding detailed follow-up, such as a survey highlighting areas
of a phenomenon than the one method would allow, for more in-depth observation and interviewing. An
for example, incorporating qualitative methods into in-depth case study involving qualitative methods may
quantitative studies like RCTs to provide more detailed be productively succeeded by a survey to explore its
information on the context in which the trial takes applicability in other contexts. Mason (2006) argues for
place, the process of the trial and what mechanisms a ‘qualitatively driven’ approach to mixing methods, as
may operate to make an intervention successful or not. qualitative ‘thinking’ can help potential mixed-methods
Debate continues about the parts that quantita- researchers to approach in creative ways the complexities
tive and qualitative methodologies can play in research of the lived experiences and social realities they seek to
and how they may take on ‘senior’ or ‘junior’ status investigate.
within projects (Murphy et al. 1998). Researchers may There are increasing numbers of examples in the
or may not place equal emphasis on the quantitative and health services research literature of ways in which
qualitative methods in the mixed-methods research they mixed-methods research has been undertaken. Perhaps
undertake. In the Priority-Sequence Model, Morgan the most significant contribution to the development
(1998) describes how in the design stage of the of mixed-methods research in this field has been the
research, priority is usually accorded to one or the introduction of qualitative research methods alongside
other, depending on whether the research question is RCTs and other quantitative studies. Indeed, the
principally quantitative or qualitative. Further consider- Medical Research Council (MRC) has recommended
ation of the research design will take into account what the implementation of qualitative research prior to
aspects of the research question may be answered more phase II (exploratory) and phase III (main) trials. In
comprehensively or in greater depth by the use of one the document ‘A Framework for Development and
of a range of complementary methods. Evaluation of RCTs for Complex Interventions to
Mixed-methods research 255
Improve Health’ (MRC 2000), it stresses the usefulness (Glogowska et al. 2000). From 1995 to 1999, an RCT
of qualitative research in answering fundamental was undertaken to evaluate the effectiveness of the
questions about complex interventions, such as why existing community-based SLT service in 16 clinics in
does an intervention appear to work, what aspects of the Greater Bristol area for children aged 3.5 years or
health professionals’ behaviour influence the interven- under experiencing speech and language delays. The
tion, what are its ‘active ingredients’, which groups of typical provision for children accessing SLT services in
participants are more likely to respond positively and that area was compared with ‘watchful waiting’ for a
how it may be necessary to modify interventions for period of 12 months. However, alongside the quantita-
certain groups of patients. Being able to ‘unpack the tive experimental design and the child-focused quantita-
black box’ of an intervention may enable it to be targeted tive outcome measures the research team acknowl-
in the optimal way at those who will benefit optimally edged the importance of, and key rôle played by
(Hawe et al. 2004). Clearly, the type of interventions the children’s parents in the process of therapy and
conducted in SLT can be categorized as ‘complex’. its outcomes. Taking a methodologically pragmatic
Therefore, when designing intervention evaluations SLT stance and adopting methods that would hopefully
researchers need to acknowledge this and build in robust answer different aspects of the questions about the
preliminary work to elucidate what is involved in the SLT provision on offer, the research team pursued the
therapy. concept of ‘complementarity’. The aim was to produce
In health services research, researchers have used a broader understanding of the phenomenon of pre-
qualitative methods to inform and improve the way an school speech and language delay and treatment, in
RCT investigating prostate cancer testing and treatment terms of its effects on both children and their parents.
was designed and conducted (Donovan et al. 2002). Thus, within the overall framework of the RCT, a
The qualitative findings pointed to the need for the parallel investigation of parents’ views, comprising a
researchers to present information to potential partici- quantitative (questionnaire survey) study and qualita-
pants differently in order for them to understand why tive (individual interview) study, was implemented.
the trial was necessary. As a result, recruitment to the trial The substantive results of the mixed-methods
became more efficient and more patients agreed to take approach are published in detail elsewhere (Glogowska
part. Campbell et al. (2003b) nested a qualitative study et al. 2002) but there was little evidence to support the
within an RCT evaluating physiotherapy for osteoarthri- effectiveness of the early SLT provision under evaluation.
tis patients to supplement clinical data with informa- Of the five primary outcome measures chosen, only
tion from patient-based outcome measures. Exploration one—receptive language—showed a statistically signifi-
of RCT participants’ understanding of the randomiza- cant improvement in the children who received therapy
tion process has taken place (Snowdon et al. 1997, compared with those who did not. Similarly, only two
Featherstone and Donovan 1999, 2002). While these of 11 secondary outcome measures showed statisti-
studies are capable of standing independently of the cally significant differences between the groups—overall
trials they took place alongside, they contribute much resolution of difficulties and improvement in phonology.
to the understanding of the process issues involved Thus the RCT produced a rather disparate picture which
in those RCTs specifically and how researchers can was hard to interpret. The questionnaires and interviews
ensure that patients give fully informed consent to had an important rôle in making sense of such findings.
their involvement in clinical trials in general. Bloor The lack of effectiveness evidenced in the RCT results
(1976) employed qualitative observational methods were echoed by the lack of difference between parents
to explore unanticipated findings from a quantita- whose children received therapy and those who did not
tive study showing geographical differences in surgery. when asked in the questionnaire whether their child’s
Oakley et al. (2006) utilized qualitative data to examine difficulties had improved.
process issues in conducting an RCT of a peer-delivered One of the key areas of interest in the research
sex education intervention and Moffatt et al. (2006) was the progress made by the pre-school children, as
explored how discrepancies between quantitative and the background literature suggested that there was a
qualitative components were handled in a pilot RCT of high rate of spontaneous resolution in children with
welfare rights advice to a population of older people. early talking difficulties. While the outcome measures
demonstrated that both groups of children made
progress over the year in which they were involved in the
Mixed methods and SLT: an example
RCT, many of them continued to experience difficulties
Mixed-methods research has also been undertaken in their late pre-school and early school years. About
in SLT. The author was part of a research team 70% of the children had not moved into the normal
which conducted a mixed-methods study of pre-school range on one or more of the speech and language
speech and language therapy, known locally as STEP outcomes at the end of the trial. The questionnaire
256 Margaret Glogowska
results and interview findings reflected this also, with the The challenges that mixed-methods research poses both
majority of parents (70%) who completed the question- for researchers personally and for their institutions
naire and ten out of 16 parents interviewed stating their cannot be ignored either.
belief that further therapy was necessary for their child.
Thus, it is evident that, while the RCT could
The primacy of the research question—or not?
provide objective information about the progress and
outcomes of the children, the questionnaire and Researchers should only implement a mixed-methods
interview components gave insight into the perceptions, approach where one is called for. This point seems
attitudes and beliefs of the parents regarding the self-evident, yet it is crucial to point out that a mixed-
interventions. These in turn helped to explain and methods approach does not always constitute a superior
set into context the equivocal findings of the RCT— choice and is not always appropriate. It is a methodol-
apparent lack of effectiveness of the provision evaluated ogy among methodologies that a researcher may use,
and the lack of resolution of many of the children’s but ultimately it is the research question that drives the
difficulties. Without the additional information from choice of methodologies and methods and not vice versa.
the complementary studies, the interpretation of the The tail (mixed-methods research) should not end up
RCT findings would have proved much harder and an wagging the dog (the research question). In some areas,
unnecessarily bleak picture of the value of pre-school a single methodology with all its inherent weaknesses,
SLT might have emerged. The integrated findings from which will need to be dealt with and accounted for in the
the three components of the study suggested a number conduct, analysis and writing up of the research, may
of areas in need of further exploration in practice and still be the best choice for answering a research question.
research, including the effects of increasing the amounts However, some authors have questioned whether doing
of therapy given, more careful targeting of treatment mixed-methods research has not become something of
and greater engagement on the part of therapists with a ‘fad’ (Bryman 2006a, O’Cathain et al. 2007b). In
parental opinions and views. The headline statement empirical studies conducted by these authors, the issue
may have been that the therapy provision had not of research funding also reared its ugly head, with
‘worked’ but the research team was able to advance some some researchers intimating their belief that funding
explanation of why this was the case and the factors bodies may look more favourably upon mixed-methods
involved. This is an example of one of the more typical research proposals. For that reason, mixed-methods
ways in which quantitative and qualitative approaches designs might represent a strategic choice on the part
have been combined in health services research but it of researchers rather than be determined by the research
is one where ‘inferences made from mixed-methods question.
studies can be greater than the parts’ (O’Cathain et al.
2007b: 148).
What is in a name?
Even a brief glance at the mixed-methods research
The challenges of doing mixed-methods literature will reveal a number of terms describing the
research use of the methods in the research process, including
At this point it may be tempting to wonder why ‘mixing’, ‘combining’ and ‘integrating’. An area of
researchers still continue to pursue single-method considerable and ongoing debate in mixed-methods
studies at all when mixed-methods studies appear to research is the nature and extent of this ‘mixing’
offer so much more in terms of both comprehensiveness of methods. Moran Ellis et al. (2006) describe the
and understanding. Whether or not one accepts that ways in which the terms ‘mixing’, ‘combining’ and
mixing methods in research is a ‘good thing’ (Mason ‘integrating’ methods have come to be used interchange-
2006), there are still considerable challenges in doing ably. They argue that these terms are not equivalent
so. These occur: and attempt to delineate the differences. Combining
methods, according to them, refers to a situation where
one method may precede or follow after another to
• at in the conceptualization stage—making a strong provide greater explanation or ‘fleshing out’. In this type
case for using mixed methods; of endeavour, one method acts as an adjunct to the other
• at the stage of operationalization—defining how but does not have an equal position with the other in
the methods will be used in conjunction with each the research.
other and rigour ensured; and Integration, on the other hand, is where different
• at the synthesis stage—how the findings of methods keep what makes them different at the
the different methods will be interpreted and paradigm level but are be interlinked with each other,
reported. to produce a fuller account of a phenomenon. This
Mixed-methods research 257
interlinking of methods may occur at different stages end up being treated (and published) separately. At the
of the research process, as early as the stage where the end of a mixed-methods research study, it can happen
study is first conceptualized and as late as the actual that researchers perceive that different audiences will
writing-up. Methods may also be integrated at any of be interested in different aspects of their study. Thus,
the stages in between these extreme points, including in they may write up quantitative findings, aiming them at
sampling, data collection and data-analysis activities— one audience, and qualitative findings, aiming them at
what Sandelowski (2000) refers to as ‘the technique another. In part, this may come about because of issues
level’. Many mixed-methods studies are integrated at the surrounding publication. Researchers may perceive that
latter stages. Thus, sampling decisions and collection journals tend to focus more strongly on either quantita-
of data may take place separately for the different tive or qualitative research and integrating quantita-
methods but are then brought together at the stage tive and qualitative findings may then leave them in a
of data analysis, interpretation and hypothesis/theory publishing ‘no man’s land’. Word restrictions in journals
generation. may also contribute to their difficulties—describing
In the SLT example of mixed-methods research both sets of methods and results in limited space may
presented in this paper, at the stage where conceptu- not be feasible. It may also be true that researchers
alization took place, the RCT was the overarching focus more on one set of findings because they have
methodology, as the primary research question was that greater confidence in them, either because of personal
of the effectiveness of the SLT services being evaluated. methodological preferences or because of their awareness
At the time, there was very little evidence about the of situational factors which my have weakened one of
effects of SLT among such young children or indeed the methods undertaken. A mixed-methods researcher
about the likelihood of their recovering spontaneously. may also find one set of findings of greater intrinsic
During the implementation of the RCT further research interest (possibly arising again out of methodological
questions regarding the perceptions of the children’s preferences) or more noteworthy than the other and
parents about speech and language delay and treatment this, too, may lead to selective writing up.
received became an important focus of attention. At Another explanation for quantitative and qualita-
that point, the research team brought other methods tive findings being reported separately is the timing of
into play, which were more appropriate for answering the components of the research. Phases of research may
this type of question. Data collection for the RCT, proceed in such a way that one component is completed
the questionnaire survey and the qualitative interviews before another and researchers may then feel under
then proceeded together. Integration of the methods pressure to write up and publish. Nor should the actual
occurred in terms of the sampling decisions made for practical difficulties in producing a synthesis of quantita-
the qualitative study. The research team wanted to tive and qualitative findings be underestimated. These
sample purposively for interview, for example to include are well encapsulated in Dixon-Woods et al. (2005),
parents from differing educational backgrounds and who examined the theoretical, conceptual and procedu-
socio-economic status; to include parents of children ral issues involved in synthesizing evidence from studies
with different combinations and severities of difficul- obtained by systematic review. Bryman (2007) also cites
ties, of different ages and of different outcomes. Data the lack of exemplars in the literature, where quantita-
from the RCT and the parent questionnaire enabled tive and qualitative methods have been conducted and
the research team to make the sampling choices that written up in an integrated way. This absence of ‘worked
would ensure a range of parents could be included examples’ may cause researchers to doubt that such a
in the interview study. A further phase of integration report would find a receptive audience and thus deter
was achieved during the collation and interpretation of them from attempting to combine and integrate their
results—although the findings of all three methods were own findings.
capable of standing alone, the research team was eager
that they should tell an integrated, more holistic story,
Telling a different story?
resulting in the aforementioned paper. At the time of
writing up, there were few other exemplars to draw on. Bryman (2006b) issues a warning that mixed-methods
research can produce an abundance and depth of data
that researchers had not anticipated. In turn, this
Telling the whole story?
can generate unpredicted outcomes and unexpected
Bryman (2007), in his qualitative interview study of a findings—a welcome state of affairs when greater
sample of social science researchers, suggests a number understanding and articulation of a phenomenon is
of reasons why researchers may not seek to integrate the achieved through it. However, there is the chance
findings from diverse methods and why quantitative and that inconsistency and conflict between findings of
qualitative components of mixed method studies may mixed-methods studies can occur (Moffatt et al. 2006,
258 Margaret Glogowska
Campbell et al. 2003b). In the mixed-methods study primarily either ‘quantitative’ or ‘qualitative’ researchers.
conducted by Moffatt et al. (2006), the implementation Their greater familiarity with, and experience of, the
of welfare rights advice and its impact on health and one may mean that they then highlight the findings
social outcomes among a population of older people of the paradigm they feel more ‘comfortable’ with, in
was evaluated. Quantitative questionnaire data revealed regard to their own skills, creating a barrier to integra-
that the advice had made little impact on the health tion of findings. This may be a particular stumbling
and social outcomes of those who had received it, block within a profession like SLT, where research
while qualitative interview data suggested that it had capacity building is an ongoing project. Bryman (2007)
had a significant positive effect. The authors revisited suggests that the incorporation of a range of research
their data, and conducted a systematic re-interrogation knowledge and skills into research teams might alleviate
of each data set. They concluded that the outcomes this problem, whereby members can deploy their own
measured in the quantitative component of the study particular strengths but at the same time play a part in
were different from those in the qualitative component. developing the skills of others. However, he recognizes
They did not argue that one data set represented any that such a move may be counter-productive. It may
‘truer’ sense of what was going on than the other and as actually facilitate the type of quantitative–qualitative
such, they recommended the reporting of both sets of split mixed-methods research is meant to overcome in
findings as complementary and a way of demonstrating pursuit of greater understanding of a research topic,
how mixed-methods research can be part of an effort to by dividing up the work into distinct quantitative
present a more thorough understanding of a complex and qualitative compartments undertaken separately by
phenomena. those with the relevant knowledge and skills. Clearly,
In Campbell et al. (2003b) discrepancies were found this creates a challenge to both individual researchers
between patients’ assessments of outcomes in an RCT and to the institutions in which they work, if they are to
of physiotherapy treatment for osteoarthritis of the design and conduct mixed-methods research, achieve
knee. The patient-based outcomes used were items real integration of findings and develop expertise in
concerning pain and function collected quantitatively by doing so.
questionnaire and qualitatively by in-depth interview-
ing. The authors reported a lack of agreement between
How shall we know them?
the questionnaire and interview findings and tried to
make sense of them. After ruling out obvious differences Another issue exercising the minds of those who have
such as the timing of the outcome measures, they written about mixed-methods research is the thorny one
concluded that the context in which the patients gave the of quality and how it can be ensured in studies compris-
researchers information was likely to have played a key ing qualitative and qualitative components. By what
rôle. The quantitative information had been collected criteria are mixed-methods studies to be judged? The
in a clinical setting, with a doctor present, while the increased attention paid to the issue of the quality of
interview accounts had been collected independently of research has been largely due to the increasing use, and
the clinic in the patients’ own homes. Mixed-methods acceptance of, qualitative research (Seale 1999). Many
research presents considerable challenges in marrying qualitative researchers have looked askance at the idea
up different ways of collecting and analysing data. of using the traditional criteria of quality in quantitative
Considerably more effort is then involved in interpret- research, such as establishing:
ing those data when they seem to be saying very different
things. However, complexity is an inherent part of many • validity (does a measure actually measure what it
health and social interventions, and taking a reduction- is supposed to measure?);
ist approach to outcomes by saying whether interven- • reliability (does a measure produce similar results
tions appear to ‘work’ or not may leave many important across time and across measurers?); or
questions unanswered. Moreover, the ‘works for whom, • generalizability (can the findings for the research
where, and in what circumstances’ information, likely sample be applied across the wider population?).
to be supplied by mixed-methods research, may provide
a more realistic, and ultimately more useful, picture of Thus, new criteria, more meaningful to qualitative
what is going on. researchers have been developed and applied such as
those of Lincoln and Guba (1985), who stress the
importance of research ‘trustworthiness’ and concepts
Individual and institutional barriers
such as:
Researchers conducting mixed-methods research, while
recognizing the need for a pragmatic approach to • credibility (do these findings make sense and can
methodology selection, may still classify themselves as I believe them?);
Mixed-methods research 259
• dependability (would these findings be repeated possibilities for SLT research and should have a vital
in a similar context?); part in it. In turn, SLT research should be able to
• confirmability/transparency (is it clear how the contribute substantively to the future development of
research took place?); or mixed-methods research.
• transferability (would the findings be relevant to
another setting?).
References
Bryman (2006a) broaches the subject of quality criteria BARBOUR, R. S., 1999, The case for combining qualitative and
and discusses the use of ‘convergent’ criteria—where the quantitative approaches in health services research. Journal
same criteria are used for both quantitative and qualita- of Health Services Research and Policy, 4, 39–43.
tive elements of a mixed-methods study; ‘divergent BLOOR, M., 1976, Bishop Berkeley and the adenotonsillectomy
criteria—where separate criteria are used for the different enigma: an exploration of variation in the social construc-
tion of medical disposals. Sociology, 10, 43–51.
elements and ‘bespoke’ criteria—where new criteria BRITTEN, N., CAMPBELL, R., POPE, C., DONOVAN, J., MORGAN, M.
are developed specifically with mixed-methods research and PILL, R., 2002, Using meta-ethnography to synthesise
in mind. Bryman recommends essentially a pragmatic qualitative research: a worked example. Journal of Health
stance, which take full account of the purpose of the Services Research and Policy, 7 (4), 209–215.
methods used. For example, it may be better to use BRYMAN, A., 1988, Quantity and Quality in Social Research (London:
Routledge).
convergent criteria in the form of validity and reliability, BRYMAN, A., 2006a, Paradigm peace and the implications for quality.
rather than separate or bespoke criteria, when qualita- International Journal of Social Research Methodology, 9 (2),
tive research is contributing to the development of a 111–126.
measurement tool such as an attitude scale. Likewise, BRYMAN, A., 2006b, Integrating quantitative and qualitative
when quantitative data in the form of demographic research: how is it done? Qualitative Research, 6 (1), 97–113.
BRYMAN, A., 2007, Barriers to integrating quantitative and qualita-
information is being used as part of a wider qualita- tive research. Journal of Mixed Methods Research, 1 (1), 8–22.
tive project, convergent criteria more geared to the CAMPBELL, R., POUND, P., POPE, C., BRITTEN, N., PILL, R.,
establishing of the research’s trustworthiness might be MORGAN, M. and DONOVAN, J., 2003a, Evaluating meta-
applicable. ethnography: a synthesis of qualitative research on lay
experiences of diabetes and diabetes care. Social Science and
Medicine, 56 (4), 671–684.
Conclusion CAMPBELL, R., QUILTY, B. and DIEPPE, P., 2003b, Discrepancies
between patients’ assessments of outcome: qualitative study
This paper has discussed the background to the develop- nested within a randomised controlled trial. British Medical
ment of mixed-methods research—paradigm war and Journal, 326, 252–253.
peace—and demonstrated the ways in which, as a DAVID, R. M., ENDERBY, P. and BAINTON, D., 1982, Treatment of
acquired aphasia: speech therapists and volunteers compared.
methodological ‘third way’ of pragmatism, it has already Journal of Neurology Neurosurgery and Psychiatry, 45, 957–
contributed to health services research and, indeed, to 961.
SLT research. It has also argued for the use of mixed- DIXON-WOODS, M., AGARWAL, S., JONES, D., YOUNG, B. and
methods research where it is appropriate and where it SUTTON, A., 2005, Synthesising qualitative and quantitative
can genuinely provide a better or more comprehen- evidence: a review of possible methods. Journal of Health
Services Research and Policy, 10 (1), 45–53.
sive, all-round, in-depth answer than use of a single DONOVAN, J., MILLS, N., SMITH, M., BRINDLE, L., JACOBY, A.,
method can. It should be clear that not all research PETERS, T., FRANKEL, S., NEAL, D. and HAMDY, F., 2002,
should be mixed method. It does not hold the mortgage Improving design and conduct of randomised trials by
on research design desirability any more than quantita- embedding them in qualitative research: ProtecT (prostate
tive or qualitative methodologies alone do. Nor is it to testing for cancer and treatment) study. British Medical
Journal, 325, 766–770.
be regarded as some kind of methodological panacea, FEATHERSTONE, K. and DONOVAN, J., 1999, Random allocation or
whereby the weaknesses of one approach are automati- allocation at random? Patients’ perspectives of participation
cally cancelled out by the strengths of another. in a randomised controlled trial. British Medical Journal, 317,
The paper has also highlighted the considerable 1177–1180.
difficulties which still beset mixed-methods research and FEATHERSTONE, K. and DONOVAN, J., 2002, Why don’t they just tell
me straight, why allocate it?’ The struggle to make sense of
the future challenges it needs to confront. However, the participating in a randomised controlled trial. Social Science
complex arena of healthcare, as it is being provided in and Medicine, 55, 709–719.
the 21st century, calls for as complete a methodolog- FITZ-GIBBON, C. T., 1986, In defence of randomised controlled
ical armoury as possible in order to tackle the many trials with suggestions about the possible use of meta-analysis.
questions about the people it serves and the services it British Journal of Disorders of Communication, 21, 117–
124.
delivers. In the same way, because of the multifaceted, GLOGOWSKA, M., CAMPBELL, R., PETERS, T. J., ROULSTONE, S.
multifactorial nature of human communication and its and ENDERBY, P., 2002, A multimethod approach to the
disorders, mixed-methods research offers a wealth of evaluation of community preschool speech and language
260 Margaret Glogowska
therapy provision. Child: Care, Health and Development, 28 MURPHY, E. and DINGWALL, R., 1998, Qualitative methods in health
(6), 513–521. services research. In N. Black, J. Brazier, R. Fitzpatrick and
GLOGOWSKA, M., ROULSTONE, S., ENDERBY, P. and PETERS, T. B. Reeves (eds), Health Services Research Methods: A Guide to
J., 2000, Randomised controlled trial of community-based Best Practice (London: BMJ Books, 129–138.
speech and language therapy for pre-school children with MURPHY, E., DINGWALL, R., GREATBATCH, D., PARKER, S. and
delayed speech and language. British Medical Journal, 321, WATSON, P., 1998, Qualitative research methods in health
923–926. technology assessment: a review of the literature. Health
GLOGOWSKA, M., ROULSTONE, S., ENDERBY, P. and PETERS, T., Technology Assessment, 2, 16 (available at: http://www.hta.
2001, RCTs: myths, misconceptions and mastery. Bulletin nhsweb.nhs.uk/fullmono/mon216.pdf).
of the Royal College of Speech and Language Therapists, 587, O’CATHAIN, A., MURPHY, E. and NICHOLL, J., 2007a, Why, and
6–7. how, mixed-methods research is undertaken in health services
HAWE, P., SHIELL, A. and RILEY, T., 2004, Complex interventions: research in England: a mixed-methods study. BMC Health
how ‘out of control’ can a randomised controlled trial be? Services Research, 7, 85.
British Medical Journal, 328, 1561–1563. O’CATHAIN, A., MURPHY, E. and NICHOLL, J., 2007b, Integra-
JOHNSTONE, P. L., 2004, Mixed methods, mixed methodology health tion and publications of ‘yield’ from mixed-methods
services research in practice. Qualitative Health Research, 14 studies. Journal of Mixed Methods Research, 1 (2), 147–
(2), 259–271. 163.
KUHN, T. S., 1970, The Structure of Scientific Revolutions, 2nd edn OAKLEY, A., 2000, Experiments in Knowing (Cambridge: Polity).
(Chicago, IL: University of Chicago Press). OAKLEY, A., STRANGE, V., BONELL, C., ALLEN, E., STEPHENSON, J.,
LINCOLN, N. B., MCGUIRK, E., MULLEY, G. P., LENDREM, W., JONES, RIPPLE STUDY TEAM, 2006, Process evaluation in randomised
A. C. and MITCHELL, J. R. A., 1984, Effectiveness of speech controlled trial of complex interventions. British Medical
therapy for aphasic stroke patients: a randomised control trial. Journal, 332,, 413–416.
Lancet, i, 1197–1200. ONG, B. N., 1993, The Practice of Health Services Research (London:
LINCOLN, Y. S. and GUBA, E., 1985, Naturalistic Inquiry (Beverly Chapman & Hall).
Hills, CA: Sage). PATTON, M., 1988, Paradigms and pragmatism. In D. Fetterman
MASON, J., 2006, Mixing methods in a qualitatively driven way. (ed.), Qualitative Approaches to Evaluation in Educational
Qualitative Research, 6 (1), 9–25. Research (Newbury Park, CA: Sage), pp. 116–137.
MEDICAL RESEARCH COUNCIL (MRC), 2000, Framework for PATTON, M., 2002, Qualitative Methods and Evaluation, 3nd edn
Development and Evaluation of RCTs for Complex Interven- (Thousand Oaks, CA: Sage).
tions to Improve Health (available at: http://www.mrc.ac.uk/ POPE, C. and MAYS, N., 1995, Reaching the parts other methods
Utilities/Documentrecord/index.htm?d=MRC003372). cannot reach: an introduction to qualitative methods in health
MEIKLE, M., WECHSLER, E., TUPPER, A., BENENSON, M., BUTLER, and health services research. British Medical Journal, 311, 42–
J., MULHALL, D. and STERN, G., 1979, Comparative trial 45.
of volunteer and professional treatments of dysphasia after PRING, T., 2004, Ask a silly question: two decades of troublesome
stroke. British Medical Journal, 2, 87–89. trials. International Journal of Language and Communication
MOFFATT, S., WHITE, M., MACKINTOSH, J. and HOWEL, D., 2006, Disorders, 39 (3), 285–302.
Using quantitative and qualitative data in health services RITCHIE, J. and LEWIS, J., 2003, Qualitative Research Practice
research—what happens when mixed method findings (London: Sage).
conflict? BMC Health Services Research, 6, 28. SANDELOWSKI, M., 2000, Combining qualitative and quantitative
MORAN ELLIS, J., ALEXANDER, V. D., CRONIN, A., DICKINSON, M., sampling, data collection, and analysis techniques in mixed-
FIELDING, J., SLENEY, J. and TOMAS, H., 2006, Triangulation method studies. Research in Nursing and Health, 23, 246–
and integration; processes, claims and implications. Qualita- 255.
tive Research, 6 (1), 45–59. SEALE, C., 1999, The Quality of Qualitative Research (London: Sage).
MORGAN, D. L., 1998, Practical strategies for combining qualitative SIEGEL, G. M. and YOUNG, M. A., 1987, Group deigns in clinical
and quantitative methods: applications to health research. research. Journal of Speech and Hearing Disorders, 52, 194–
Qualitative Health Research, 8 (3), 362–376. 199.
MORGAN, D. L., 2007, Paradigms lost and pragmatism regained: SNOWDON, C., GARCIA, J. and ELBOURNE, D., 1997, Making sense
methodological implications of combining qualitative and of randomization: responses of parents of critically ill babies
quantitative methods. Journal of Mixed Methods Research, 1 to random allocation of treatment in a clinical trial. Social
(1), 48–76. Science and Medicine, 45 (9), 1337–1355.