See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/260228377

Defining and operationalising L2 complexity

Chapter · October 2012

DOI: 10.1075/lllt.32.02bul

CITATIONS READS

384 4,959

2 authors:

Bram Bulté Alex Housen

Vrije Universiteit Brussel Vrije Universiteit Brussel
44 PUBLICATIONS 1,190 CITATIONS 98 PUBLICATIONS 3,363 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Alex Housen on 15 January 2015.

The user has requested enhancement of the downloaded file.

John Benjamins Publishing Company

This is a contribution from Dimensions of L2 Performance and Proficiency.

Complexity, Accuracy and Fluency in SLA.
Edited by Alex Housen, Folkert Kuiken and Ineke Vedder.
© 2012. John Benjamins Publishing Company
This electronic file may not be altered in any way.
The author(s) of this article is/are permitted to use this PDF file to generate printed copies to
be used by way of offprints, for their personal use only.
Permission is granted by the publishers to post this file on a closed server which is accessible
to members (students and staff) only of the author’s/s’ institute, it is not permitted to post
this PDF on the open internet.
For any other use of this material prior written permission should be obtained from the
publishers or through the Copyright Clearance Center (for USA: www.copyright.com).
Please contact [email protected] or consult our website: www.benjamins.com
Tables of Contents, abstracts and guidelines are available at www.benjamins.com
 chapter 2

Defining and operationalising L2 complexity

Bram Bulté & Alex Housen

Vrije Universiteit Brussel

This chapter takes a critical look at complexity in L2 research. We demonstrate

several problems in the L2 literature in terms of how complexity has been defined
and operationalised as a construct. In the first part of the chapter we try to
unravel its highly complex, multidimensional nature by presenting a taxonomic
model that identifies major types, dimensions and components of L2 complexity,
each of which can be independently analysed or measured. The second part
evaluates how complexity has been measured in empirical SLA research. Using
the taxonomy of L2 complexity from part 1 as a framework, we inventory the
measures of L2 complexity that have been used in a sample of forty recent L2
studies. Next we discuss the construct validity of several widely used measures
of grammatical complexity by identifying their underlying logic as well as the
methodological and practical challenges that their computation presents.

1. Complexity in SLA research

In current SLA research, two broad strands can be distinguished in which the
­construct of complexity plays an important role (Housen & Kuiken 2009). In
the first strand, complexity figures as an independent variable, that is, as a factor
whose influence on some aspect of L2 performance or L2 proficiency is investi-
gated. One example is research on the effects of instruction on SLA, where several
studies have looked at the impact of the complexity of the target structure on its
teachability or on the effectiveness of different kinds of instruction (e.g. DeKeyser
1998; Doughty & Williams 1998; Housen, Van Daele & Pierrard 2005; Spada &
Tomita 2010).
In the second line of research, complexity is investigated as a ­dependent
variable, often together with fluency and accuracy, as a basic descriptor of
­
L2 ­performance and as an indicator of L2 proficiency. Here, the complexity of L2
learners’ performance and proficiency is measured in order to demonstrate the
effect of some other variable (such as age or other learner variables, different types

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

of learning contexts, or different kinds of instruction) on L2 attainment (e.g. Bygate

1996, 1999; Derwing & Rossiter 2003; Collentine 2004; Norris & Ortega 2000).
Studies that have used complexity as a dependent and/or independent
­variable have produced mixed and sometimes contradictory results (cf. Robinson
2007; ­Skehan 2009; Spada & Tomita 2010). We think that these inconsistent results
can at least partly be accounted for by the way in which L2 complexity has been
defined and operationalised. Many L2 studies that investigate ‘complexity’ either
do not define what they mean by this term, or when they do, they do so in general,
vague or even circular terms. This is illustrated by the following examples:
(1) “[complexity is the] use of more challenging and difficult language …
­Complexity is the extent to which learners produce elaborated language”
 (R. Ellis & Barkhuizen 2005: 139)
(2) “Grammatical and lexical complexity mean that a wide variety of both basic
and sophisticated structures and words are available to the learner”
 (Wolfe-Quintero, Inagaki, & Kim 1998: 69, 101)
(3) “Complexity refers to … the complexity of the underlying interlanguage
system developed” (Skehan 2003: 8)

Given such general definitions it is not surprising to observe a wide range of ­different
interpretations across and within studies of what constitutes L2 c­ omplexity. A more
explicit characterization of complexity is needed to correctly interpret the results of
complexity measurements and to draw felicitous inferences and conclusions about
the role or nature of the independent variables investigated in L2 research.

2. Defining complexity

Despite its prominent position in contemporary science (Mitchell 2009), there is no

commonly accepted definition of complexity. Its etymological Latin origins (from
com ‘together’ + plectere ‘to braid’) are reflected in general dictionary definitions
such as “consisting of many different and connected parts” (New Oxford North
American Dictionary) and “the state or quality of being intricate or complicated;
hard to separate, analyze or solve” (Merriam-Webster Unabridged Dictionary).
More elaborate characterizations of complexity include that of Rescher (1998: 1), a
philosopher, who defines complexity as “a matter of the quantity and variety of the
constituent elements [of an item] and of the interrelational elaborateness of their
organizational and operational make-up”, and who identifies more than ten ‘modes
of complexity’. Suffice it for here to say that, at the most basic level, complexity
refers to a property or quality of a phenomenon or entity in terms of (1) the number
and the nature of the discrete components that the entity consists of, and (2) the
number and the nature of the relationships between the constituent components.
© 2012. John Benjamins Publishing Company
All rights reserved
 Defining and operationalising L2 complexity 

In the language sciences we can distinguish a relative and an absolute approach

to the notion of complexity (Dahl 2004; Miestamo, Sinnemäki & Karlsson 2008).
Both relative and absolute complexity refer to properties of language features
(i.e. items, patterns, constructions, rules), of (sub-)systems thereof, or of the uses
to which these features are put (see upper-left part of Figure 1).

L2 complexity

Relative complexity Absolute complexity

(Difficulty) (Complexity)

Subjective Objective
determinants determinants

Discourse-interactional Propositional
Linguistic complexity complexity complexity

System complexity Structure complexity

Formal Functional
complexity complexity

Lexical Morphological Syntactic Phonological

Collocational Inflectional Sentence Segmental

Lexemic Derivational Clausal Suprasegmental

Phrasal

Figure 1. A taxonomy of complexity constructs

The relative approach defines complexity in relation to language users: a lan-

guage feature or system of features is seen as complex if it is somehow costly or
taxing for language users and learners, particularly in terms of the mental effort
or resources that they have to invest in processing or internalizing the feature(s).
Thus, relative complexity, or cognitive complexity or simply difficulty as we will
call it, refers to the mental ease or difficulty with which linguistic items are
learned, processed or verbalized in the processes of language acquisition and use
(­Hulstijn & De Graaff 1994). For instance, psycholinguistic studies have found
that certain embedded structures (e.g. relative clauses) and passives are harder to
process, or emerge later in language acquisition, than other structures (e.g. coor-
dinate and active structures) (Byrnes & Sinicrope 2008; Diessel 2004). As further
shown in Figure 1, the difficulty of a language feature is to some extent subjec-
tive or learner-dependent. A language feature which is costly, difficult or hard for
some learners or users may be less costly, less hard or even easy for other learners,
© 2012. John Benjamins Publishing Company
All rights reserved
 Bram Bulté & Alex Housen

depending on such individuality factors as their level of L2 development, language

aptitude, memory capacity, L1 background, motivation, and so forth. Apart from
these subjective, learner-dependent factors, there are also more objective, learner-
independent factors that can contribute to the ease or difficulty with which L2
features are learned and processed. These objective factors include the perceptual
saliency and frequency of occurrence of L2 features in the input (Goldschneider &
DeKeyser 2001), their communicative load and, as indicated by the dotted line in
Figure 1, also their absolute or inherent complexity, or complexity for short.
The absolute approach defines language complexity in objective, ­quantitative
terms as the number of discrete components that a language ­feature or a ­language
system consists of, and as the number of connections between the ­different
­components. It follows then, that difficulty is a broader notion than i­nherent
­complexity, which is only one of the factors that may contribute to ­learning or
processing difficulty. It also follows that there is not necessarily a o ­ ne-to-one
relationship between the inherent complexity of a language feature and its
­
­processing or learning difficulty (Rohdenburg 1996). In the remainder of this
­section, we will focus on the notion of inherent complexity as it has been applied
to the characterization of L2 performance and L2 proficiency.

3. L2 complexity

For the heuristic purposes of analyzing learners’ L2 performance and L2 proficiency

in SLA research,1 we argue that the broader notion of L2 complexity minimally con-
sists of three components: propositional complexity, ­discourse-interactional com-
plexity and linguistic complexity (see Figure 1). Briefly, propositional complexity
refers to the number of information or idea units which a speaker/writer encodes
in a given language task to convey a given message content (Zaki & R. Ellis 1999;
R. Ellis & Barkhuizen 2005). For instance, the L2 performance of a speaker who
encodes 55 idea units in narrating a story or in describing a picture will be proposi-
tionally more complex than that of a speaker who only encodes 25 idea units.
Discourse-interactional complexity is still a vague concept. It has mainly been
proposed in analyses of learners’ dialogic discourse, where the d
­ iscourse-interactional
complexity of learners’ L2 performance has been c­haracterized in terms of the

1. While acknowledging that the relationship between the language performance (production,
comprehension) of L2 learners and their L2 proficiency and underlying interlanguage systems
is obviously a complicated one, language performance (esp. production) is seen here as the
concretization of the L2 knowledge and ability of a language learner.

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

number and type of turn changes that learners initiate and the interactional moves
and participation roles that they engage in (e.g. Duff 1986; Gilabert, Barón & Llanes
2009; Pallotti 2008).
Propositional complexity and discourse-interactional complexity are still
­relatively new notions that have received far less attention in the L2 literature
than ­linguistic complexity. Linguistic complexity has been interpreted in the
L2 ­literature in two different ways: either as a dynamic property of the learner’s L2
system at large (global or system complexity), or as a more stable property of the
­individual linguistic items, structures or rules that make up the learner’s L2 system
(local or structure complexity). Global or system complexity refers to the degree
of e­ laboration, the size, breadth, width, or richness of the learner’s L2 system or
‘repertoire’, that is, to the number, range, variety or diversity of different structures
and items that he knows or uses: whether he masters a small or a wide range of
different words or different grammatical structures, whether he controls all or only
a fraction of the sound system of the L2, and so forth.
When we look at linguistic complexity at the local level of the individual
­linguistic features themselves, we speak of their structure complexity. Structure
complexity has more to do with depth than with breadth or range. As shown in
Figure 1, structural complexity itself can be further broken down into distinct sub-
types, such as the formal and functional complexity of an L2 feature (­DeKeyser
2005; Doughty & Williams 1998; Housen et al. 2005). Functional complexity refers
to the number of meanings and functions of a linguistic structure and to the degree
of transparency, or multiplicity, of the mapping between the form and meanings/
functions of a linguistic feature. With some structures there is straightforward,
one-to-one mapping of meaning onto form (e.g. English plural marker -s). Such
structures are functionally less complex than structures where there is no such
isomorphism between form and function/meaning (e.g. the syncretic grammati-
cal marker 3sg Present -s in English, or polysemic lexical items such as ­English
‘­present’). Formal complexity can refer to a number of things, including the struc-
tural ‘substance’ of a linguistic feature as determined by the number of discrete
components of a linguistic form (e.g. simple past vs. present perfect forms in
­English). Formal complexity has also been defined in terms of the number of oper-
ations to be applied on a base structure to arrive at the target s­ tructure (e.g. in the
derivation of passive clauses from underlying active structures). Finally, some have
argued that formal complexity has to do with the dependency distance between a
form and its nearest head or dependent (e.g. the plural -s form in ­English, which
is locally determined within the NP versus the 3sg Present -s, which is globally
determined outside the VP in which it occurs).
Figure 2 further shows that the different sub-dimensions of linguistic com-
plexity distinguished here can be evaluated across various language domains

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

(­ phonology, lexis, morphology, syntax) and their respective subdomains

(e.g. inflectional morphological and derivational morphological complexity;
phrasal, clausal and sentential syntactic complexity). We can thus speak of the
global elaborateness, or systemic complexity, of the learner’s L2 phonological
­system, lexical system, morphological system, and so on. Alternatively, we can
look at the local functional and formal complexity of the individual syntactic,
morphological, phonological or lexical features that make up the learner’s L2
system.
It is hoped that the breakdown of the complexity construct outlined above
may serve as a descriptive-analytic framework for future analyses of L2 complex-
ity, allowing researchers to be more specific as to what they mean when they
state that they investigate ‘L2 complexity’. Still, several remarks are in order. First,
the different types of complexity distinguished here are distinct constructs in
­theory only. In the reality of language use and learning, several of these complex-
ity constructs may be closely intertwined, which complicates their identification
and assessment. Secondly, the model presented here is merely a taxonomy, not
a ­theory of complexity. The need for such a theory is illustrated by the fact that
­different authors use different criteria to distinguish between structurally simple
and complex features, which has lead to contradictory characterizations and clas-
sifications of the same feature. A telling example in case is the 3sg Present -s
in English (Housen et al. 2005; Spada & Tomita 2010), which has been variably
characterized as a formally and functionally simple feature (Krashen 1994), a for-
mally simple yet functionally complex feature (R. Ellis 1990) and as a formally
and functionally complex feature (DeKeyser 1998). Such contradictory treat-
ments of the same linguistic feature clearly demonstrate the need for a linguisti-
cally or theoretically motivated metric of linguistic and particularly structural
complexity.
Equally worrisome is that most L2 studies (at least the ones reviewed for this
chapter) do not specify or define the type of complexity construct that they are
investigating at all. Rather, complexity is merely operationalised in the sense that
it is specified in terms of quantitative measures only, such as the subordination
ratio, the Guiraud Index or the mean length of utterance (MLU). This situation is
illustrated in Figures 2 and 3, which provide a schematic overview of the different
levels of construct specification (Bachman 2005) for two major components of
‘linguistic complexity’, grammatical complexity and lexical complexity (see also
Bulté, Housen, Pierrard & Van Daele 2008).
The construct of linguistic complexity can be examined on at least three dif-
ferent levels. First, complexity can be analysed on an abstract theoretical level as
a property of a (cognitive) system and/or of a structure (that forms part of such a

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

c­ ognitive system) in terms of its number of components, the degree of embedded-

ness of these components, and in terms of the relationships that exist between them.
On a more concrete, observational level of language performance, as e­ xemplified
by a sample of actual language use, these theoretical notions of c­ omplexity can
be manifested in language behaviour in various ways and on several different
­levels (e.g. in the use of different strategies for combining and embedding clauses,
by using different verb forms or specialized versus more common vocabulary).
Finally, there is the level of the analytical measures and instruments that have been
designed to give a concrete (quantitative) indication of the degree of complex-
ity of a given language sample, so that the complexity of different samples can
be analyzed and compared more objectively. These measures are situated on the
operational level. The distinction between these three different levels (theoretical,
observational, operational) is an important one that has to be made explicit, since
failing to do so results in the danger of reducing complexity to merely one of its
operationalisations (e.g. the subclause ratio). Also, in order to have measures that
tap into linguistic complexity in a meaningful and valid way, it has to be estab-
lished first what complexity is (theoretical), how it is or can be manifested in actual
language performance (observational), and how these behavioural manifestations
can be somehow captured or quantified (operational). The links between these dif-
ferent levels of construct specification should be as transparent as possible in order
for meaningful research interpretations to be made.
Figure 2 further shows that two major sources of grammatical complexity can
be distinguished, syntactic and morphological complexity, each further divisible
into even smaller and finer-grained subcomponents.

Theoretical level Systemic grammatical complexity Structural grammatical complexity

of (elaboration, size, range, variation, (‘depth’, embeddedness, compositionality
cognitive constructs: of grammatical L2 structures)
‘breadth’ of L2 grammar)
Grammatical complexity

Observational level Grammatical diversity Grammatical sophistication

of
behavioural constructs:
Surface manifestations of Syntactic complexity Morphological complexity
grammatical complexity Sentence Clausal Phrasal Derivational
in L2 performance Inflectional
complexity complexity complexity complexity complexity

Operational level Length Subclause syntatic dependents/ verb derivational

of measures ratios arguments/ phrasal head inflections/ affixes/
statistical constructs: (words/T-unit; (simple and clause verbs word
Measures words/clause) adjusted) number of
(frequency, ratio, index) verb forms

Figure 2. Grammatical complexity at different levels of construct specification

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

Theoretical level Systemic lexical complexity Structural lexical complexity

of (elaboration, size, range, breadth (‘depth’ of L2 lexical items
cognitive constructs: of repertoire of L2 lexical & collocations)
items & collocations)
Lexical complexity

Observational level
of
behavioural constructs:
Density Diversity Compositionality Sophistication
Surface manifestations of
lexical complexity in L2
performance

Operational level Lexical words/ Type/token ratios Morphemes/ Frequency-based

of function words (e.g. Guiraud, Über, words (‘advanced’)
statistical constructs: Herdan), D type/token
Syllables/words ratios (e.g. lexical
Number of
Measures (lexical) word frequency profile;
(frequency, ration, index) type (or lemmas) lambda)

Figure 3. Lexical complexity at different levels of construct specification

Lexical measures (Figure 3) can be said to tap into at least three different
aspects of lexical complexity, the density, diversity and sophistication of lexical
performance (Skehan 2003; Bulté et al. 2008), to which we want to add a fourth
aspect, the compositionality of lexical elements, that is, the number of formal and
semantic components of lexical items (e.g. phonemes, morphemes, denotations).2
As indicated above, most L2 studies define linguistic complexity at the l­owest
level of construct specification only, as an operational-statistical construct. Only
a few s­tudies have attempted to define lexical and grammatical complexity as
behavioural constructs (Ortega 2003; Norris & Ortega 2009; Bulté et al. 2008;
­Skehan 2003; Skehan & Foster 2005). To our knowledge, complexity has rarely
been adequately defined in the CAF literature at a more theoretical level as a
­cognitive construct. This is ­problematic. As is shown in Figures 2 and 3, the rela-
tionship between operational, behavioural and theoretical-cognitive constructs
is by no means straightforward (as indicated by the multiple lines between the
constructs at the different levels of construct specification). Therefore a clear spec-
ification of what is meant by grammatical and lexical complexity as theoretical
­constructs is necessary, if only to establish the construct validity of the complexity
­measures employed in empirical research (which, as we will demonstrate below,
is ­sometimes moot). But in addition to this, there are also problems in the CAF
­literature at the level of complexity measurement, to which we turn next.

2. Figures 2 and 3 present a necessarily simplified picture of the full multidimensionality of

the grammatical and lexical complexity construct. Further sub-dimensions and sub-constructs
can be conceived, for example, for lexical complexity (e.g. lexemic vs. collocational complexity).

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

4. A survey of complexity measurement

The complexity of L2 performance and proficiency has been evaluated in the CAF
­literature by means of a wide variety of tools, ranging from holistic and ­subjective
­ratings by lay or expert judges, to more objective quantitative measures of L2
­production (R. Ellis & Barkhuizen 2005; Wolfe-Quintero et al. 1998). Most L2
studies, however, opt for quantitative, objective measures and therefore these will
be the focus of what follows.
In this section we first examine how complexity has been measured, and what
type or component of complexity has been measured, across a representative
sample of forty empirical L2 studies on task-based language learning published
between 1995 and 2008 (indicated with an asterisk in the reference section). A first
observation is that there is no shortage of complexity measures in SLA studies.
This is clear from Table 1, which presents an inventory of the complexity measures
used in our sample of studies, loosely classified in terms of the behavioural and
theoretical complexity constructs which they gauge (cf. Figures 2 and 3) – ‘loosely’,
and tentatively, because, as we will argue below, many of these measures are hybrid
measures which simultaneously tap into several sub-components and subdomains
of L2 complexity.3
A second observation is that some sub-components or sources of linguistic
complexity are well-covered by a wide range of measures (especially sentential
syntactic complexity through subordination, and lexical diversity), whereas other
types or sources of linguistic complexity are covered by only one or two measures
(e.g. lexical density and sophistication) or have not been measured at all (e.g. deri-
vational morphological complexity, phrasal syntactic complexity, collocational
lexical complexity). Table 1 suggests possible measures for these uncovered com-
ponents, preceded by ‘Ø’.4 The picture that emerges from Table 1 is reinforced by
Table 2, which shows the distribution of the different complexity measures used in
each of the 40 studies across the various types, domains and sources of grammati-
cal and lexical complexity (horizontal numbers refer to the 40 measures as listed
in Table 1; vertical numbers refer to the studies in the sample listed underneath
Table 2).

3. For the same reason, and for the sake of surveyability, no further sub-classifications are
made in Table 1 between measures of systemic, structural, formal or functional complexity.
4. Not all uncovered types of linguistic complexity are included in Table 1, for practical
reasons. For instance, the constructs of collocational lexical complexity and phonological
complexity are not included in Table 1 as they have never been directly measured, neither in
our sample nor in any other CAF study that we know of.

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

There are a number of general observations that can be made. First, most
­studies use general measures which tap global, overarching complexity constructs
(e.g. mean number of words per T-unit (1), mean number of clauses per T-/c-/
AS-unit (studies 8, 9, 10)), rather than fine(r)-grained measures that tap specific
features of the learners’ lexical and grammatical L2 systems and performance
(e.g. number of relative clauses per T-unit (16), or measures 20–27 in Table 1).

Table 1. Inventory of linguistic complexity measures in task-based studies. (Possible

measures for subcomponents not covered by the studies in the sample are suggested and
indicated with Ø).
A. GRAMMATICAL COMPLEXITY

  a. Syntactic
    i. Overall
      1. Mean length of T-unit
      2. Mean length of c-unit
      3. Mean length of turn
      4. Mean length of AS-unit
      5. Mean length of utterance
      6. S-nodes / T-unit
      7. S-nodes / AS-unit
    i. Sentential – Coordination
      Ø Coordinated clauses / clauses
ii.
     Sentential – Subordination
      8. Clauses / AS-unit
      9. Clauses / c-unit
      10.  
Clauses / T-unit
      11.  
Dependent clauses / clause
      12.  
Number of Subordinate clauses
      13.  
Subordinate clauses / clauses
      14.  Subordinate clauses / dependent clauses
      15.  
Subordinate clauses / T-unit
      16.  
Relative clauses / T-unit
      17.  
Verb phrases / T-unit
    iii. Subsentential (Clausal + Phrasal)
      18.  
Mean length of clause
      19.  
S-nodes / clause
(Continued)

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

Table 1. (Continued)

    iv. Clausal

      Ø Syntactic arguments / clause
    v. Phrasal
      Ø Dependents / (noun, verb) phrase
    vi. Other (± syntactic sophistication)
      20.  
Frequency of passive forms
      21.  
Frequency of infinitival phrases
      22.  
Frequency of conjoined clauses
      23.  
Frequency of Wh-clauses
      24.  
Frequency of imperatives
      25.  
Frequency of auxiliaries
      26.  
Frequency of comparatives
      27.  
Frequency of conditionals
  b. Morphological
    i. Inflectional
      28. Frequency of tensed forms
      29. Frequency of modals
      30. Number of different verb forms
      31. Variety of past tense forms
    ii. Derivational
     Ø Measure of affixation

B. LEXICAL COMPLEXITY
  a. Diversity
     32. Number of word types
    33. TTR
    34. Mean segmental TTR
    35. Guiraud Index
    36. (Word types)2 / words
    37. D
  b. Density
     38. Lexical words / Function words
     39. Lexical words / Total words
  c. Sophistication
     40. Less frequent words / Total words

© 2012. John Benjamins Publishing Company

All rights reserved
 
Table 2. Measures of linguistic complexity used in 40 L2 studies (task-based learning; 1995–2008) (Measure numbers refer to the m
­ easures
as listed in Table 1)
Grammatical Lexical Total
Syntactic Morphological
Overall Sentential - Subordination Subsentential Other Inflectional Diversity Density Sophist
Measure 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

All rights reserved

Study
1 X 1
Bram Bulté & Alex Housen

2 X X X 3
3 X 1
4 X 1
5 X X X 3
6 X X X X 4
7 X X 3
8 X X 2
9 X 1

© 2012. John Benjamins Publishing Company

10 X X 2
11 X 1
12 X X 2
13 X X X X X X X X X 9
14 X 1
15 X X X X X 5
16 X X X X 4
17 X X X X 4
18 X X X X 4
19 X 1
20 X X 2
21 X X X X 4

Index of studies:
1: Albert & Kormos (2004); 2: Bygate (1996); 3: Bygate (2001); 4: Elder & Iwashita (2005); 5: R. Ellis & Yuan (2004); 6: R. Ellis & Yuan (2005); 7: Foster (1996); 8: Foster & Skehan (1996); 9: Gass
et al. (1999); 10: Gilabert (2007); 11: Guará-Tavares (2008); 12: Isbell et al. (2005); 13: Ishikawa (2007); 14: Iwashita et al. (2001); 15: Iwashita et al. (2008); 16: Kawauchi (2005); 17: Kuiken et al.
(2005); 18: Kuiken & Vedder (2007); 19: Lambert & Engler (2007); 20:Mehnert (1998); 21: Michel et al. (2007).
 Grammatical Lexical Total
Syntactic Morphological Lexemic
Overall Sentential - Subordination Subsentential Other Inflectional Diversity Density Sophist
Measure 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

All rights reserved

Study
22 X X X 3
23 X X X X 4
24 X 1
25 X 1
26 X X 2
27 X* X X 3
28 X* X X 3
29 X X X X X X X 7
30 X X 2
31 X X X X X X 6

© 2012. John Benjamins Publishing Company

32 X X X X 4
33 X 1
34 X 1
35 X* X X X X X 6
36 X X 2
37 X X X 3
38 X 1
39 X 1
40 X X X 3
Total 5 3 1 1 5 5 1 7 9 12 5 3 2 1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 1 3 2 2 9 2 6 1 1 3 3 2

* Used as a measure of fluency.

Index of studies (contd.):
22: Mochizuki & Ortega (2008); 23: Ortega (1995); 24: Ortega (1999); 25: Rahimpour & Yaghoubi (2007); 26: Révész(2008); 27: Robinson (1995); 28: Robinson (2001); 29: Robinson (2007);
30: Rutherford (2001); 31: Sangarun (2005); 32: Sercu et al. (2006); 33: Skehan & Foster (1997); 34: Skehan & Foster (2005); 35: Storch & Wigglesworth (2007); 36: Tajima (2003); 37: Tavakoli & Foster
(2008); 38: Tavakoli & Skehan (2005); 39: Wigglesworth (1997); 40: Yuan & Ellis (2003).
Defining and operationalising L2 complexity

 Bram Bulté & Alex Housen

­ obinson and N. Ellis (2008) have recently called for the use of more specific
R
­measures focusing on individual grammatical phenomena such as conjunctions
or verb phrase morphology to complement the use of global complexity measures,
a call to which some researchers have already responded (Michel 2010; Révész
2009; Robinson, ­Cadierno & Shirai 2009).
Second, and more importantly, in most studies that claim to measure L2
­complexity only a few measures are calculated (the mean number of measures is
2.7, with 22 studies using one or two measures only). This is probably due to the
lack of adequate computational tools for automatic complexity measurement and
the ­labour-intensiveness of manual computation.5
Third, a small number of measures are used in many studies (e.g. measures
9, 10, 33) while the majority of measures in Table 2 is used in a few studies only
(and 19 measures are each used in one study only). As a result, both within and
across CAF studies, only a limited range of what constitutes linguistic com-
plexity is c­ overed: mainly lexical diversity and/or syntactic sentential complex-
ity through subordination. Other sources of grammatical complexity and other
­sub-components of ­lexical complexity or any aspect of collocational complexity,
are rarely measured, if at all.
At the same time, several studies often repeatedly, and probably redundantly,
measure the same sub-construct of linguistic complexity more than once, by using
what may well be merely variants of the same measure gauging the same under-
lying observational and theoretical complexity construct. Study 35, for example,
calculated four different measures of syntactic subordination.
The last two findings corroborate similar observations by Norris and Ortega
(2009). In general, we can say that empirical CAF research has taken a rather
narrow, reductionist, perhaps even simplistic view on and approach to what con-
stitutes L2 complexity. Still, on the basis of these limited measurement practices,
general claims are made not only about learners’ L2 complexity in general (or
even about their L2 proficiency at large), but also about the effects of the inde-
pendent variables under investigation (such as the effectiveness of specific task
manipulations or specific instructional methods). This is problematic, particu-
larly in the light of the questionable construct validity and underlying logic of
some of the most popular complexity measures, which we discuss in the next
section.

5. This situation may well change in the near future with the advance of online complexity
analyzers (e.g. http://www.personal.psu.edu/xxl13/). However, the ease of computation
­afforded by such automatic complexity tools does not obviate users from asking themselves
exactly what these tools measure and how.

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

5. A closer look at syntactic complexity measures

A great number of complexity measures are currently at the disposal of L2

­researchers (cf. Table 1 for a non-exhaustive sample). Each measure has its own
strengths but also presents challenges in terms of reliability, validity, sensitivity
and discriminatory power and, not in the least, its practical feasibility. Some of
these issues have been documented in the literature (cf. Norris & Ortega 2009;
Ortega 2003; Pallotti 2009; Wolfe-Quintero et al. 1998) but several others still
merit further attention. Previous discussions of validity have mainly focused on
concurrent validity. In this section we will pay particular attention to the construct
validity of complexity measures, that is, the extent to which measures adequately
represent their underlying behavioural and theoretical constructs. As Norris and
Ortega (2009) have urged, “[w]e really need to establish interpretation-centered
warrants for what our measures purportedly are measuring” (p. 570). Building
from the case of a few common measures of syntactic complexity6 we will point
to concerns about their adequacy both as metrics of L2 syntactic performance and
proficiency and as metrics of L2 syntactic development.
Syntactic complexity measures typically aim to quantify one or more of the
following: range of syntactic structures, length of unit, degree of structural com-
plexity (‘sophistication’) of certain syntactic structures and amount and type of
coordination, subordination and embedding.
A first observation is that many of the complexity measures are ambiguous
or hybrid measures in that they simultaneously capture not one but several differ-
ent, potentially independent and unrelated behavioural or theoretical complexity
constructs and sources of complexity. This hybrid quality is reflected in Figures 2
and 3 by the multiple lines connecting the measures (statistical constructs) and the
behavioural constructs.
Length-based metrics of syntactic complexity (e.g. measures 1–5 in Table 1)
are an example in case. Length measures capture the mean length of a certain
unit of analysis in terms of the number of words or morphemes.7 As such they

6. See Bulté et al. (2008) for a discussion of issues pertaining to the construct validity of
lexical complexity measures.
7. Instead of the more common expression ‘mean length of utterance’ we prefer the term
‘mean length of unit’ as a cover term for the following units of production and analysis: clause,
sentence, T-unit, AS-unit. Definitions and discussions of the suitability of the various types
of production units can be found in Bardovi-Harlig (1992), R. Ellis and Barkhuizen (2005,
Chapter 7), Byrnes, Maxim, and Norris (2010) and Foster, Tonkyn, and Wigglesworth (2000).
The two most frequently used measures in the CAF literature are probably the T-unit and the
clause. The general remarks about mean length measures formulated here hold, in principle,
for any of the aforementioned units of analysis.

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

­ easure syntactic complexity in the sense of structural substance or composition-

m
ality. But particularly when they are calculated as the number of morphemes per
production unit these measures not only inform about the structural syntactic com-
plexity but also about the morphological (inflectional, derivational) complexity of a
language sample. Furthermore, these length measures also simultaneously tap into
different layers of syntactic structure and different sources of complexity – phrasal,
clausal and sentential (Norris & Ortega 2009; cf. below). For this very reason, length
measures are often used in L2 research, as in L1 research, as generic measures of
overall syntactic or grammatical complexity or even of linguistic ­proficiency in
general (e.g. Iwashita, Brown, McNamara & O’Hagan 2008 and Tavakoli & Foster
2008 for L2 research; Brown 1973 & Hunt 1965 for L1 research).
Most measures of syntactic subordination (8–15 in Table 1) are also hybrid
measures, though in a different sense than length measures. They not only cap-
ture linguistic (syntactic) diversity, depth and compositionality but also ‘difficulty’.
The reason why subordinate structures are singled out or given a greater weight
in the measurement of syntactic complexity is because they are purportedly cog-
nitively harder to process than other types of syntactic linking (Lord 2002; Bygate
1999). A similar assumption underlies many of the remaining syntactic complex-
ity measures in Table 1 (e.g. 16, 20–27) as well as other complexity indices which
assign different weights to different syntactic structures to reflect putatively differ-
ent degrees of difficulty (e.g. the Syntactic Complexity Formula, Botel, Dawkins &
Granowski 1973; the Elaboration Index, Loban 1976). In terms of the framework
of language complexity presented in Section 2, difficulty is a distinct construct
from structural complexity, and the correspondence between the two constructs
still has to be demonstrated rather than a priori assumed, and there is no guaran-
tee that it holds for all syntactic structures (Pallotti 2009).
The consequence of the discussion so far is that it is important that we
­motivate our complexity measures by stating what particular type, component
or sub-­construct of complexity they represent and, in the case of hybrid and
generic ­measures, by explaining how the different measures for one conglomorate
­complexity construct interact.
Specifying the level or domain of complexity targeted by syntactic complexity
measures is also necessary for determining their adequacy as valid, reliable and
sensitive indicators of syntactic (or more general linguistic) growth and develop-
ment. We will consider the developmental facets of the two most common types of
syntactic complexity measures, length measures and subordination measures. But
before we turn to these issues, two general remarks are in order. First, linguistic
complexity measures cannot be validated simply by showing that they increase
in the course of acquisition. Developmental timing may give an indication of the
difficulty of a grammatical construction or subsystem, but even if so, difficulty is

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

conceptually distinct from linguistic complexity. Whether, or to what extent, struc-

tural complexity increases over time needs to be established empirically rather
than be taken for granted.
Secondly, any measure that serves as an index of development would probably
have to cover the full trajectory of language acquisition, from the lowest level or stage
to the highest. Such a quality may come at a price, however. Measures that distinguish
between samples at broad stages of development may not be sensitive enough
to ­discriminate between samples at one stage or between pretest and posttest
­samples for one group. Also, change is not necessarily progress and may not reflect
­improvement or more target-like behaviour (Pallotti 2009). Furthermore, as ­studies
conducted in a dynamic systems framework have shown, various ­developmental
factors may interact and several sub-dimensions of CAF may compete, leading to
nonlinear, U-shaped trends for some measures (Larsen-Freeman 2006; Verspoor,
Lowie & Van Dijk 2008).
The available studies on length measures in L2 acquisition show mixed results.
Some researchers have argued that they have been proven to be reliable and valid
when it comes to measuring syntactic L2 development in broad linguistic strokes
because they have been found to develop linearly with increasing proficiency level
and to correlate with standardised tests (Larsen-Freeman & Strom 1977; Wolfe-
Quintero et al. 1998). Others, however, have argued that such results follow from
circular argumentation where MLU is included in how the different proficiency
levels are defined (Dewaele 2000; Unsworth 2008). Also, if the situation in L1
development is any indication, the suitability of length measures as indicators
of L2 growth across the entire developmental spectrum is likely to be limited.
Although MLU has a broad concurrent validity in L1 development that persists up
to ages five-nine (Rice, Redmond & Hoffman 2006), it plateaus beyond a score of
4.0 words (Miles & Bernstein Ratner 2001). Similar plateaus probably also exist in
L2 development and may be reached even sooner than in L1 development because
L2 learners are often capable of producing multi-morpheme and multi-word
utterances almost immediately from the onset of acquisition (­Larsen-Freeman &
Strom 1977: 124).
The limitations of subordination measures as indices of syntactic or of more
­general grammatical development mainly stem from their specific and fairly
narrow linguistic scope. Subordination indices target one of the three main
levels of syntactic organisation only – the sentential level, not the clausal and
phrasal level – and they tap into one source of syntactic complexity only, namely
embedding through subordination. Other sources of syntactic c­ omplexity, such
as clausal coordination or nominalisation and modification at the phrasal level
are not gauged by subordination measures. For this r­ eason, researchers have
also questioned the suitability of subordination ratios as indicators of syntactic

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

growth across the full range of L2 ­development (R. Ellis & Barkhuizen 2005;
Norris & Ortega 2009; Ortega 2003; ­Verspoor et al. 2008; ­Wolfe-Quintero et al.
1998). Norris and Ortega (2009) point out that, at the early stages of L2 devel-
opment, ­syntactic ­complexity is first established through coordination. Only at
later, ­intermediate stages does subordination become the dominant means of
syntactic complexity, while at even more advanced stages of L2 development,
syntactic complexity would be mainly achieved through increasing complexity
at the phrasal level. Norris and Ortega therefore argue that, in addition to mea-
sures of subordination, research should also include measures of coordination
and phrasal complexity. The specific measures which Norris and Ortega recom-
mend to this end are, r­ espectively, the Coordination Index and the Mean Length
of Clause. Since Norris and Ortega (2009) has become a standard citation in
CAF research, their argumentation and recommendations warrant careful con-
sideration. As we attempt to demonstrate below, the two measures which they
propose are problematic, both in their calculation and in their interpretation.
The Coordination Index (CI) was developed by Bardovi-Harlig (1992) as a
measure of syntactic complexity of L2 learners’ written production and is calculated
by dividing the number of coordinated clauses in a language sample by the total
number of ‘combined clauses’. These combined clauses consist of both coordinated
and subordinated clauses. Therefore, the CI cannot be viewed as a ‘pure’ measure
of clause coordination, but rather as a measure of clause subordination, since, ulti-
mately, the score on this index depends on the amount of subordination produced.
This is illustrated in Table 3. Each sample contains two coordinated clauses but their
scores on the CI are substantially different (1.0 vs. 0.67). Clearly, this is misleading.
What we want is a measure that yields a high score when many coordinated clauses
are produced, and a low score when few coordinated clauses are produced, inde-
pendently of the amount of subordination. Instead of, or as a complement to the CI,
it is advisable to use alternative measures of ­coordination – for example measures
whose mathematical logic is analogous to that of the ­subclause ratio, that is, by
dividing the number of coordinated clauses by the number of sentences (or T-units
or AS-units) produced (see last column in Table 3).
Turning to measures of phrasal complexity, Norris and Ortega recommend
the mean number of words per clause (mean length of clause = MLC). The under-
lying idea is that since the number of phrases in a clause is limited, increases in
clause length will reflect increases in phrase length (e.g. through modification of
the head). Again, two remarks are in order here.
First, the MLC cannot be considered a ‘pure’ measure of phrasal ­complexity.
Clause length increases not only through expansion at the phrasal level (e.g. via
pre- or post-modification) but also through expansion at the clausal level, for
example, by adding adjuncts (of time, manner, place).

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

Table 3. Coordination in two text samples

Clause Clause Coordination Coordinated
coordinations subordinations Index clauses/­
sentence
Sample A
1. A boy has a frog.
2. The frog is in a jar.
3. In the night the frog goes
away.
4. The boy wakes up and his 1
frog is gone away.
5. He looks everywhere but 1
he can’t find his frog.
Total 2 1.00 0.40
Sample B
1. A boy has a frog.
2. The frog is in a jar.
3. In the night the frog goes
away.
4. The boy wakes up and he 1 1
sees that his frog is gone
away.
5. He looks everywhere but 1
he can’t find his frog.
Total 2 1 0.67 0.40

Secondly, clause length is crucially determined by how one defines and

operationalises a clause. There is a lack of agreement across studies in this
respect, which may cause differences in findings and in interpretations (Ishikawa
1995; Polio 2001; Unsworth 2008). Some researchers (e.g. Van Daele et al. 2008;
Kuiken & Vedder, this volume) use a linguistic definition of a clause, as a unit
consisting of a subject (visible or implied) plus a predicate, i.e. a construction
with a finite or nonfinite predicator or verb as its head (e.g. Jackson 2008). Such a
definition not only has the benefit of being linguistically valid but also of acknowl-
edging a range of nonfinite clausal constructions, thus respecting the linguistic
integrity of advanced learner samples. However, there are some disadvantages to
using a linguistic definition of a clause when c­ alculating clause length, at least for
the purpose of capturing phrasal ­complexity. Applying such a linguistic defini-
tion implies that verb constructions such as ‘go look’, ‘tries finding’, ‘keeps shouting’

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

and ‘starts to look’ are all analyzed as consisting of two clauses, a main clause plus
a nonfinite subordinate complement clause. Consequently, learners who produce
nonfinite subordinate clauses will receive lower MLC values than learners who
do not, irrespective of the actual length or complexity of the constituent phrases.
This probably explains why some studies treat verb clusters with a non-finite verb
complement as a single verb phrase that heads a single main clause rather than
two clauses. Still others follow Hunt’s (1965) working definition of a clause as a
unit which requires “a visible subject and a finite verb” (p. 29). The advantage of
Hunt’s definition is that the identification of finite verbs is fairly straightforward,
and amenable to automatic coding, which increases the feasibility of computa-
tion. The downside is that the scope of the MLC is narrowed to finite clauses
with a surface subject, thereby excluding nonfinite constructions and clauses with
subject ellipsis.
Clearly, none of the measurement problems discussed here are insurmount-
able. But, ideally, we should employ measures of intra-phrasal complexity that are
independent of other layers and other sources of syntactic complexity. Alternative
measures to this end that we are currently investigating include the number of
dependents per phrasal head and the number of words per phrase (especially in
noun phrases) (Bulté in preparation).
More generally, as shown in the first part of this chapter, there is now ample
evidence that syntactic complexity, as a sub-component of linguistic complexity,
is itself a multilayered construct consisting of distinct sub-constructs that relate
to different sources of complexity which each must be gauged by different mea-
sures (e.g. Ortega 2003; Spoelman & Verspoor 2010). For this reason Norris and
Ortega (2009) advocate an “organic” approach to the study of complexity that
employs multivariate research designs. They recommend to measure complexity
not only with generic measures such as length measures but also with specific
“distinct and complementary” (p. 562) complexity measures that capture phrasal
complexity and sentence complexity via s­ ubordination as well as coordination in
addition to the diversity and sophistication of structures produced (pp. 561–562).
What we have tried to show in this section is that none of the complexity mea-
sures employed or recommended in the L2 research is unproblematic, neither in
its computation nor in its interpretation.

6. Conclusion

This chapter critically scrutinized a number of issues involved in the definition

and operationalisation of complexity as a basic dimension of L2 performance, pro-
ficiency and development in CAF-based research.

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

We have tried to explicate that language complexity is a multifaceted,

multidimensional and multilayered construct, a fact that is still insufficiently
acknowledged in L2 research. Language complexity has cognitive and linguistic
dimensions, and performance and developmental facets, and can manifest itself
at all levels of language structure and use. Next to the problems of construct defi-
nition (the fact that complexity in CAF research lacks adequate definitions sup-
ported by theories of linguistics, cognition or language learning), there are also
problems concerning its operationalisation, that is, how complexity has been, and
can be, validly, reliably and efficiently measured in empirical research.
Our survey of current complexity measurement practices in CAF research,
as exemplified by Task-based Language Learning, revealed various problems, not
only in terms of the analytic challenges which these measures present or in terms
of their reliability and sensitivity, but also in terms of their validity. This observa-
tion corroborates what other authors have claimed (e.g. Norris & Ortega 2009;
Pallotti 2009; Polio 2001; Wolfe-Quintero et al. 1998), and it limits the generaliz-
ability of the complexity results of previous CAF studies, and of the conclusions
based on these results.
In our opinion, the link between theoretical characterizations of complexity
and the way in which complexity has been operationalised in CAF research has
not been explicit enough. The concept of complexity has been used mainly in an
intuitive manner and more time has been spent on developing new measures of
language complexity than on thinking about what complexity in language actually
entails. Therefore, in addition to critically evaluating the complexity measures that
have been used so far, and in addition to conducting meta-analyses of previous
CAF studies, more fundamental research is also necessary into the nature and
the manifestation of complexity in L2 use and L2 development. It is ultimately on
the basis of such research that the measurement of complexity in SLA needs to be
based.

References

(Entries with an asterisk are included in the sample surveyed in Sections 4 and 5).
*Albert, A., & Kormos, J. (2004). Creativity and narrative task performance: An exploratory
study. Language Learning, 54, 277–310.
Bachman, L. (2005). Statistical analysis for language assessment. Oxford: Oxford University Press.
Bardovi-Harlig, K. (1992). A second look at T-unit analysis: Reconsidering the sentence. TESOL
Quarterly, 26, 390–395.
Botel, M., Dawkins, J., & Granowski, A. (1973). A syntactic complexity formula. In
W.H. M­ acGinitie (Ed.). Assessment problems in reading (pp. 77–86). Newark DE: Interna-
tional Reading Association.

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

Brown. R. (1973). A first language. Harvard: Harvard University Press.

Bulté, B., Housen, A., Pierrard, M., & Van Daele, S. (2008). Investigating lexical proficiency
development over time: the case of Dutch-speaking learners of French in Brussels. Journal
of French Language Studies, 3(18), 277–298.
*Bygate, M. (1996). Effects of task repetition: Appraising the developing language of learners.
In J. Willis, & D. Willis (Eds.). Challenge and change in language teaching (pp. 136–146).
London: Heinemann.
Bygate, M. (1999). Quality of language and purpose of task: pattern of learners’ language on two
oral communication tasks. Language Teaching Research, 3(3), 185–214.
*Bygate, M. (2001). Effects of task repetition on the structure and control of oral language. In
M. Bygate, P. Skehan, & M. Swain (Eds.). Researching pedagogic tasks, second language
learning, teaching and testing (pp. 23–48). London: Longman.
Byrnes, H., & Sinicrope, C. (2008). Advancedness and the development of relativization in L2
­German: A curriculum-based longitudinal study. In L. Ortega, & H. Byrnes (Eds.). The
longitudinal study of advanced L2 capacities (pp. 109–138). New York: Routledge.
Byrnes, H., Maxim, H., & Norris, J.M. (2010). Realizing advanced foreign language writ-
ing ­ development in collegiate education: Curricular design, pedagogy, assessment
[­Monograph]. The Modern Language Journal, 94, Suppl. s1.
Collentine, J. (2004). The effects of learning contexts on morphosyntactic and lexical develop-
ment. Studies in Second Language Acquisition, 26(2), 227–248.
Dahl, Ö. (2004). The growth and maintenance of linguistic complexity. Amsterdam: John
Benjamins.
DeKeyser, R. (1998). Beyond focus on form: Cognitive perspectives on learning and practicing
second language grammar. In C. Doughty, & J. Williams (Eds.). Focus on form in classroom
language acquisition (pp. 42–63). New York: Cambridge University Press.
DeKeyser, R.M. (2005). What makes learning second-language grammar difficult? A review of
issues. Language Learning, 55, Suppl. 1, 1–25.
Derwing, T.M., & Rossiter, M.J. (2003). The effects of pronunciation instruction on the accuracy,
­fluency, and complexity of L2 accented speech. Applied Language Learning, 13, 1–17.
Dewaele, J.-M. (2000). Saisir l’insaisissable? Les mesures de longueur d’énoncés en linguistique
appliquée. International Review of Applied Linguistics, 38, 17–33.
Diessel, H. (2004). The acquisition of complex sentences. Cambridge: Cambridge University Press.
Doughty, C., & Williams, J. (1998). Pedagogical choices in focus on form. In C. Doughty, &
­ illiams (Eds.). Focus on form in classroom second language acquisition (pp. 197–261).
J. W
New York: ­Cambridge University Press.
Duff, P. (1986). Another look at interlanguage task: Taking task to task. In R. Day (Ed.). Talking
to learn (pp. 147–181). Rowley, Mass.: Newbury House.
*Elder, C., & Iwashita, N. (2005). Planning for test performance: Does it make a difference? In
R. Ellis (Ed.). Planning and task performance in a second language (pp. 219–237). Amster-
dam: John Benjamins.
Ellis, R. (1990). Instructed second language acquisition. Oxford: Blackwell.
Ellis, R., & Barkhuizen, G. (2005). Analysing learner language. Oxford: OUP.
*Ellis, R., & Yuan, F. (2004). The effects of planning on fluency, complexity, and accuracy in sec-
ond language narrative writing. Studies in Second Language Acquisition, 26, 59–84.
*Ellis, R., & Yuan, F. (2005). The effects of careful within-task planning on oral and written
task ­performance. In R. Ellis (Ed.). Planning and task performance in a second language
(pp. 167–192). Amsterdam: John Benjamins.

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

*Foster, P. (1996). Doing the task better: How planning time influences students’ performance.
In J. Willis, & D. Willis (Eds.). Challenge and change in language teaching (pp. 126–135).
London: Heinemann.
*Foster, P., & Skehan, P. (1996). The influence of planning on performance in task-based learn-
ing. Studies in Second Language Acquisition, 18(3), 299–324.
Foster, P., Tonkyn, A., & Wigglesworth, G. (2000). Measuring spoken language: A unit for all
reasons. Applied Linguistics, 21(3), 354–375.
*Gass, S., Mackey, A., Fernandez, M., & Alvarez-Torres, M. (1999). The effects of task repetition
on linguistic output. Language Learning, 49, 549–580.
*Gilabert, R. (2007). The simultaneous manipulation of task complexity along planning time
and (+/- Here-and-Now): Effects on L2 oral production. In M. Garcia Mayo (Ed.). Investi-
gating tasks in formal language learning (pp. 44–68). Clevedon: Multilingual Matters.
Gilabert, R., Barón, J., & Llanes, À. (2009). Manipulating cognitive complexity across task types
and its impact on learners’ interaction during oral performance. International Review of
Applied Linguistics, 47, 367–395.
Goldschneider, J., & DeKeyser, R. (2001). Explaining the ‘natural order of L2 morpheme acqui-
sition’ in English: A meta-analysis of multiple determinants. Language Learning, 51(1),
1–50.
*Guará-Tavares, M.G. (2008). Pre-task Planning, Working Memory Capacity and L2 Speech
Performance. Ph.D. dissertation, Universidade Federal de Santa Catarina, Brazil.
Housen, A., & Kuiken, F. (2009). Complexity, accuracy and fluency in second language acquisi-
tion. Applied Linguistics, 30(4), 461–473.
Housen, A., Van Daele, S., & Pierrard, M. (2005). Rule complexity and the effectiveness of explicit
grammar instruction. In A. Housen, & M. Pierrard (Eds.). Investigations in instructed sec-
ond language acquistion (pp. 235–270). Berlin: Mouton de Gruyter.
Hulstijn, J.H., & De Graaff, R. (1994). Under what conditions does explicit knowledge of a sec-
ond language facilitate the acquisition of implicit knowledge? A research proposal. AILA
Review, 11, 97–112.
Hunt, K.W. (1965). Grammatical structures written at three grade levels. NCTE research report,
no. 3. Champaign, IL: National Council of Teachers of English.
*Isbell, R.S., Sobol, J., Lindauer, L., & Lowrance, A. (2004). The effects of storytelling and story
­reading on the oral language complexity and story comprehension of young children. Early
Childhood Education Journal, 32, 157–163.
Ishikawa, S. (1995). Objective measurement of low-proficiency EFL narrative writing. Journal of
Second Language Writing, 4, 51–69.
*Ishikawa, T. (2007). The effect of manipulating task complexity along the [+/− Here-and-Now]
dimension on L2 written narrative discourse. In M. P. García Mayo (Ed.). Investigating tasks
in formal language learning (pp. 136–156). Clevedon: Multilingual Matters.
*Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S. (2008). Assessed levels of second
­language speaking proficiency: How distinct? Applied Linguistics, 29, 24–49.
*Iwashita, N., McNamara, T., & Elder, C. (2001). Can we predict task difficulty in an oral profi-
ciency test? Exploring the potential of an information-processing approach to task design.
Language Learning, 51, 401–436.
Jackson, H. (2008). Key Terms in Linguistics. London: Continuum.
*Kawauchi, C. (2005). The effects of strategic planning on the oral narratives of learners with low
and high intermediate L2 proficiency. In R. Ellis (Ed.). Planning and task performance in a
second language (pp. 143–164). Amsterdam: John Benjamins.

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

Krashen, S. (1994). The input hypothesis and its rivals. In N. Ellis (Ed.). Implicit and explicit
learning of languages (pp. 45–77). London: Academic Press.
*Kuiken, F., Mos, M., & Vedder, I. (2005). Cognitive task complexity and second language
­writing performance. In S. Foster-Cohen, M.P. García-Mayo, & J. Cenoz (Eds.). Eurosla
Yearbook. Vol. 5 (pp. 195–222). Amsterdam: John Benjamins.
*Kuiken, F., & Vedder, I. (2007). Cognitive task complexity and linguistic performance in French
L2 writing. In M.P. García Mayo (Ed.). Investigating tasks in formal language learning
(pp. 117–135). Clevedon: Multilingual Matters.
*Lambert, C.P., & Engler, S. (2007). Information distribution and goal orientation in second
language task design. In M.P. García Mayo (Ed.). Investigating tasks in formal language
learning (pp. 27–43). Clevedon: Multilingual Matters.
Larsen-Freeman, D. (2006). The emergence of complexity, fluency, and accuracy in the oral and
written production of five Chinese learners of English. Applied Linguistics, 27, 590–619.
Larsen-Freeman, D., & Strom, V. (1977). The construction of a second language acquisition
index of development. Language Learning, 27, 123–134.
Loban, W. (1976). Language development: Kindergarten through grade twelve. NCTE Research
Report No. 18. Urbana, 111. National Council of Teachers of English.
Lord, C. (2002). Are Subordinate Clauses More Difficult? In J. Bybee & M. Noonan (Eds.).
Complex sentences in grammar and discourse: Essays in honor of Sandra A. Thompson
(pp. 223–234). Amsterdam: John Benjamins.
Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. Interna-
tional Journal of Corpus Linguistics, 15(4), 474–496.
*Mehnert, U. (1998). The effects of different lengths of time for planning on second language
performance. Studies in Second Language Acquisition, 20, 52–83.
Michel, M.C. (2010). Cognitive and interactive aspects of task-based performance in Dutch as a
second language. Unpublished Ph.D. dissertation, Universiteit van Amsterdam.
*Michel, M.C., Kuiken, F., & Vedder, I. (2007). The influence of complexity in monologic
­versus dialogic tasks in Dutch L2. International Review of Applied Linguistics in Language
­Teaching, 45, 241–259.
Miestamo, M., Sinnemäki, K., & Karlsson, F. (Eds.). (2008). Language complexity: Typology, con-
tact, change. Amsterdam: John Benjamins.
Miles, S., & Bernstein Ratner, N. (2001). Parental language input to children at a stuttering
onset. Journal of Speech, Language & Hearing Research, 44, 1116–1130.
Mitchell, M. (2009). Complexity - A guided tour. Oxford: OUP.
*Mochizuki, N., & Ortega, L. (2008). Balancing communication and grammar in beginning-
level foreign language classrooms: A study of guided planning and relativization. Language
Teaching Research, 12, 11–37.
Norris, J.M., & Ortega, L. (2000). Effectiveness of L2 instruction: a research synthesis and
­quantitative meta-analysis. Language Learning, 50(3), 417–528.
Norris, J.M., & Ortega, L. (2009). Measurement for understanding: An organic approach to inves-
tigating complexity, accuracy, and fluency in SLA. Applied Linguistics, 30(4), 555–578.
*Ortega, L. (1995). The effect of planning in oral narratives by adult learners of Spanish
(Research Note No. 15). Honolulu, HI: University of Hawai’i, Second Language Teaching
and Curriculum Center.
*Ortega, L. (1999). Planning and focus on form in L2 oral performance. Studies in Second
­Language Acquisition, 21, 109–148.

© 2012. John Benjamins Publishing Company

All rights reserved
 Defining and operationalising L2 complexity 

Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A

research synthesis of college-level L2 writing. Applied Linguistics, 24(4), 492–518.
Pallotti, G. (2008). Defining and assessing interactional complexity: An empirical study. Paper
presented at AILA, Essen, August 2008.
Pallotti, G. (2009). CAF: Defining, refining and differentiating constructs. Applied Linguistics,
30(4), 590–601.
Polio, C. (2001). Research methodology in second language writing: The case of text-based stud-
ies. In T. Silva, & P. Matsuda (Eds.). On second language writing (pp. 91–116). Mahwah, NJ:
­Lawrence Erlbaum Associates.
*Rahimpour, M., & Yaghoubi-Notash, M. (2007). Examining gender-based variability in task-
prompted, monologic L2 oral performance. The Asian EFL Journal, 9(3), 156–179.
Rescher, N. (1998). Complexity: A philosophical overview. London: Transaction Publishers.
*Révész, A. (2008). Task complexity, focus on form-meaning connections, and individual differ-
ences: A classroom-based study. Paper presented at AILA, Essen, August 2008.
Révész, A. (2009). Task complexity, focus on form, and second language development. Studies in
Second Language Acquisition, 31(3), 437–470.
Rice, M., Redmond, S., & Hoffman, L. (2006). Mean length of utterance in children with
­Specific ­Language Impairment and in younger control children shows concurrent valid-
ity and stable and parallel growth trajectories. Journal of Speech, Language, and Hearing
Research, 49, 793–808.
*Robinson, P. (1995). Task complexity and second language narrative discourse. Language
Learning, 45, 99–140.
*Robinson, P. (2001). Task complexity, task difficulty, and task production: Exploring interac-
tions in a componential framework. Applied Linguistics, 22, 27–57.
*Robinson, P. (2007). Task complexity, theory of mind, and intentional reasoning: Effects on
L2 speech production, interaction, uptake and perceptions of task difficulty. International
Review of Applied Linguistics, 45, 237–257.
Robinson, P., Cadierno, T., & Shirai, Y. (2009). Time and motion: Measuring the effects of the
conceptual demands of tasks on second language speech production. Applied Linguistics,
30, 533–554.
Robinson, P. & Ellis, N.C. (2008). Cognitive linguistics, second language acquisition and L2
instruction – Issues for research. In P. Robinson & N.C. Ellis (Eds.). Handbook of cognitive
linguistics and second language acquisition (pp. 489–546). New York: Routledge.
Robinson, P., & Ellis, N. (Eds.). (2008). Handbook of cognitive linguistics and second language
acquisition. London: Routledge.
Rohdenburg, G. (1996). Cognitive complexity and increased grammatical explicitness in
­English. Cognitive Linguistics, 7, 149–182.
*Rutherford, K. (2001). An investigation of the effects of planning on oral production in a second
language. MA Thesis, University of Auckland.
*Sangarun, J. (2005). The effects of focusing on meaning and form in strategic planning.
In R. Ellis (Ed.). Planning and task performance in a second language (pp. 111–142).
­Amsterdam: John Benjamins.
*Sercu, L., De Wachter, L., Peters, E., Kuiken, F., & Vedder, I. (2006). The effect of task c­ omplexity
and task conditions on foreign language development and performance. Three empirical
studies. ITL, International Journal of Applied Linguistics, 152, 55–84.
Skehan, P. (2003). Task-based instruction. Language Teaching, 36, 1–14.

© 2012. John Benjamins Publishing Company

All rights reserved
 Bram Bulté & Alex Housen

Skehan, P. (2009). Modelling Second Language Performance: Integrating complexity, accuracy,

fluency and lexis. Applied Linguistics, 30(4), 510–532.
*Skehan, P., & Foster, P. (1997). Task type and task processing conditions as influences on for-
eign language performance. Language Teaching Research, 1, 185–211.
*Skehan, P., & Foster, P. (2005). Strategic and on-line planning: The influence of surprise infor-
mation and task time on second language performance. In R. Ellis (Ed.). Planning and task
performance in a second language (pp. 193–216). Amsterdam: John Benjamins.
Spada, N., & Tomita, Y. (2010). Interactions between type of instruction and type of language
feature: A meta-analysis. Language Learning, 60(2), 1–46.
Spoelman, M., & Verspoor, M. (2010). Dynamic patterns in development of accuracy and com-
plexity: A longitudinal case study in the acquisition of Finnish. Applied Linguistics, 31,
532–553.
*Storch, N., & Wigglesworth, G. (2007). Writing tasks: The effects of collaboration. In
M.P. ­García Mayo (Ed.). Investigating tasks in formal language learning (pp. 157–177).
­Clevedon: ­Multilingual Matters.
*Tajima, M. (2003). The effects of planning on oral performance of Japanese as a foreign language.
Ph.D. dissertation, Perdue University.
*Tavakoli, P., & Foster, P. (2008). Task design and second language performance: The effect of
narrative type on learner output. Language Learning, 58, 439–473.
*Tavakoli, P., & Skehan, P. (2005). Strategic planning, task structure, and performance t­esting.
In R. Ellis (Ed.). Planning and task performance in a second language (pp. 239–273).
­Amsterdam: John Benjamins.
Unsworth, S. (2008). Comparing child L2 development with adult L2 development: How to
measure L2 proficiency. In E. Gavruseva, & B. Haznedar (Eds.). Current trends in child
second language acquisition (pp. 301–336). Amsterdam: John Benjamins.
Van Daele, S., Housen, A. & Pierrard, M. (2008). Fluency, accuracy and complexity in the mani-
festation and development of two second languages. In S. Van Daele, A. Housen, F. Kuiken,
M. ­Pierrard, & I. Vedder (Eds.). Complexity, accuracy and fluency in second language use,
learning and teaching (pp. 301–316). Wetteren: Universa Press.
Verspoor, M., Lowie, W., & Van Dijk, M. (2008). Variability in second language development
from a dynamic systems perspective. Modern Language Journal, 92(2), 214–231.
*Wigglesworth, G. (1997). An investigation of planning time and proficiency level on oral test
discourse. Language Testing, 14(1), 21–44.
Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing:
Measures of fluency, accuracy, and complexity. Honolulu, HI: University of Hawaii, Second
Language Teaching & Curriculum Center.
*Yuan, F., & Ellis, R. (2003). The effects of pre-task and on-line planning on fluency, complexity
and accuracy in L2 monologic oral production. Applied Linguistics, 24(1), 1–27.
Zaki, H., & Ellis, R. (1999). Learning Vocabulary through interacting with written text. In
R. Ellis (Ed.). Learning a second language through interaction (pp. 151–169). Amsterdam:
John Benjamins.

© 2012. John Benjamins Publishing Company

All rights reserved
View publication stats