Structural equation

modeling
o
Rex B Kline
Concordia

D1
QICSS Set D CFA models
Resources
o Bollen, K. A., & Hoyle, R. H. (2012). Latent variable
models in structural equation modeling. In R. H.
Hoyle (Ed.), Handbook of structural equation
modeling (pp. 56–67). New York: Guilford.
o Fabrigar, L. R., & Wegener, D. T. (2012). Exploratory
factor analysis. New York: Oxford University Press.
o Kline, R. B. (2013b). Exploratory and confirmatory
factor analysis. In Y. Petscher & C. Schatsschneider
(Eds.), Applied quantitative analysis in the social
sciences (pp. 171–207). New York: Routledge.

D2
EFA
o Phases:

1. Specification
2. Extraction
3. Retention
4. Rotation
D3
Extraction methods
1. Principle components
analysis (PCA)
2. Principle axis factoring
(PAF)
3. Alpha factoring
4. ML factoring

D4
 PCA

X1 X2 X3 X4 X5 X6

A B

D5
 PAF
E1 E2 E3 E4 E5 E6

X1 X2 X3 X4 X5 X6

A B

D6
Indicator variance
Unique

Common Specific Error

Systematic

1 − rXX

D7
EFA
o Retention:

No need to specify

But best by theory

D8
EFA
o Retention:

Parallel analysis

Scree plots

D9
 4

Eigenvalue
2

1 2 3 4 5 6 7 8

Factor

D10
EFA
o Rotation:

1. Orthogonal

2. Oblique

D11
EFA
o Orthogonal:

1. Varimax
2. Quartimax
3. Equamax
D12
EFA
o Oblique:

1. Promax

2. Oblimin

D13
EFA
o Rotation:

Infinite

Not identified

D14
a) EFA (unrestricted; rotation)

E1 E2 E3 E4 E5 E6

X1 X2 X3 X4 X5 X6

A B

b) CFA (restricted; no rotation)

E1 E2 E3 E4 E5 E6
1 1 1 1 1 1

X1 X2 X3 X4 X5 X6

1 1

A B D15
CFA after EFA
o Does not “confirm” EFA:

Restricted vs. unrestricted

Items are “noisy”
Follow EFA with EFA
D16
CFA after EFA
o Osborne, J. W., & Fitzpatrick, D. C. (2012). Replication
analysis in exploratory factor analysis: What it is
and why it makes your analysis better. Practical
Assessment, Research & Evaluation, 17.
Retrieved from http://pareonline.net/pdf/
v17n15.pdf
o van Prooijen, J.-W., & van der Kloot, W. A. (2001).
Confirmatory analysis of exploratively obtained
factor structures. Educational and Psychological
Measurement, 61, 777–792.

D17
D18
 EG EA EE
1 1 1

Gender Age Ethnic

Background

D19
 E1 E2 E3
1 1 1

X1 X2 X3

+
1 −

D20
CFA specification
o Standard model:

Continuous indicators (X)

A→X←E

D21
Reflective measurement

X =T+E
2 2 2
σ =σ +σ
X T E
2
σ T
rXX = 2
σ X
D22
Reflective measurement

1− rXX
but rXX estimates a single
source

D23
CFA specification
o Standard model:

Independent E

A B

D24
CFA specification
o Unidimensional:

Simple indicator (A → X only)

No Ei Ej

D25
CFA specification
o Unidimensional:

Precise test
Convergent validity
Discriminant validity
D26
CFA specification
o Multidimensional:

Complex indicator

Ei Ej

D27
CFA specification
o Ei Ej:

Indicators share something

Repeated measures

D28
CFA specification
o Multidimensional caution:

Increases complexity

“Cheap” way to improve fit

D29
CFA specification
o Special variations:

Hierarchical CFA

MTMM models

D30
 E1 E2 E3 E4 E5 E6 E7 E8 E9
1 1 1 1 1 1 1 1 1

X1 X2 X3 X4 X5 X6 X7 X8 X9

1 1 1

Visual-
Verbal Memory
Spatial
1 1 1
DVe DVS DMe
1

D31
 Method 1 Method 2 Method 3

1 1 1

X1 X2 X3 X4 X5 X6 X7 X8 X9

1 1 1

Trait 1 Trait 2 Trait 3

D32
 E1 E2 E3 E4 E5 E6 E7 E8 E9
1 1 1 1 1 1 1 1 1

X1 X2 X3 X4 X5 X6 X7 X8 X9

1 1 1

Trait 1 Trait 2 Trait 3

D33
CFA specification
o Eid, M., Nussbeck, F. W., Geiser, C., Cole,
D. A., Gollwitzer, M., & Lischetzke, T.
(2008). Structural equation modeling of
multitrait-multimethod data: Different
models for different types of methods.
Psychological Methods, 13, 230–253.

D34
CFA identification
o Necessary:

dfM ≥ 0

Scale each latent

D35
 Scale E

ULI constraint:

D36
 Scale factor

1. Reference (marker) variable

ULI = 1, unstandardized

2. Standardize factors
UVI = 1

3. Effects coding
AVE = 1, all same metric
D37
 E1 E2 E3 E4 E5 E6
1 1 1 1 1 1

X1 X2 X3 X4 X5 X6

1 1

A B

D38
 E1 E2 E3 E4 E5 E6
1 1 1 1 1 1

X1 X2 X3 X4 X5 X6

1 A B 1

D39
 E1 E2 E3
1 1 1

X1 X2 X3

λ1 λ2 λ3
A

λ1 + λ 2 + λ 3
=1
3

D40
λ1 + λ 2 + λ 3
=1
3

λ1 = 3 − λ 2 − λ 3
λ 2 = 3 − λ1 − λ 3
λ 3 = 3 − λ1 − λ 2

D41
CFA identification
o Counting parameters:

1. Exog: Vars. + Covs.

2. Endog: Direct effects

D42
CFA identification
o Standard models:

1 factor, ≥ 3 indicators
≥ 2 factors, ≥ 2 indicators
But…
D43
CFA identification
o Nonstandard models:

No single heuristic
Undecidable
Ambiguous status
D44
TABLE 6.1. Identification Rule 6.6 for Nonstandard Confirmatory Factor Analysis Models with
Measurement Error Correlations

For a nonstandard CFA model with measurement error correlations (Rule 6.6)
to be identified, all three of the conditions listed next must hold:

For each factor, at least one of the following must hold: (Rule 6.6a)

1. There are at least three indicators whose errors are uncorrelated with each other.
2. There are at least two indicators whose errors are uncorrelated and either
a. the errors of both indicators are not correlated with the error term of a third
indicator for a different factor, or
b. an equality constraint is imposed on the loadings of the two indicators.

For every pair of factors, there are at least two indicators, one from (Rule 6.6b)
each factor, whose error terms are uncorrelated.

For every indicator, there is at least one other indicator (not necessarily (Rule 6.6c)
of the same factor) with which its error term is not correlated.

D45
 (c) (d)

EX1 EX2 EX3 EX4 EX1 EX2 EX3 EX4

1 1 1 1 1 1 1 1

X1 X2 X3 X4 X1 X2 X3 X4

1 1 1 1

A B A B

For each factor, at least one of the following must hold: (Rule 6.6a)

1. There are at least three indicators whose errors are uncorrelated with each other.

2. There are at least two indicators whose errors are uncorrelated and either

a. the errors of both indicators are not correlated with the error term of a third
indicator for a different factor, or

b. an equality constraint is imposed on the loadings of the two indicators.

D46
TABLE 6.2. Identification Rule 6.7 for Multiple Loadings of Complex Indicators in Nonstandard
Confirmatory Factor Analysis Models and Rule 6.8 for Error Correlations of Complex Indicators

Factor loadings
For every complex indicator in a nonstandard CFA model: (Rule 6.7)

In order for the multiple factor loadings to be identified, both

of the following must hold:
1. Each factor on which the complex indicator loads must satisfy
Rule 6.6a for a minimum number of indicators.
2. Every pair of those factors must satisfy Rule 6.6b that each
factor has an indicator that does not have an error correlation
with a corresponding indicator on the other factor of that pair.
Error correlations
In order for error correlations that involve complex indicators (Rule 6.8)
to be identified, both of the following must hold:
1. Rule 6.7 is satisfied.
2. For each factor on which a complex indicator loads, there must be
at least one indicator with a single loading that does not have an
error correlation with the complex indicator.

D47
CFA estimates
o Unstandardized:

1. Indicators loadings (B)

2. Factor, error variances
3. Factor, error covariances
D48
CFA estimates
o Standardized:

1. Indicators loadings (r, b)

2. Proportion unexplained
3. Factor, error correlations
D49
CFA estimates
o Failure to converge:

1. Data matrix (NPD)

2. Poor start values
3. Small N, 2 ind./factor
D50
CFA estimates
o Heywood cases (inadmissible):

1. Error variance < 0

2. | r or R2 | > 1.0
3. NPD parameter matrix
D51
 EFa EMo EFM EPr EIn
1 1 1 1 1

Father-
Father Mother Problems Intimacy
Mother

1 1

Family of Marital
Origin Adjustment

D52
Group 2: Wives

THETA-DELTA

problems intimacy father mother fa_mo

-------- -------- -------- -------- --------
problems 520.305
(130.844)
3.977
intimacy - - -27.093
(104.927)
-0.258
father - - - - 32.147
(29.214)
1.100
mother - - - - 9.967 63.416
(26.870) (28.138)
0.371 2.254
fa_mo - - - - - - - - 97.049
(25.232)
3.846

Squared Multiple Correlations for X - Variables

problems intimacy father mother fa_mo

-------- -------- -------- -------- --------
0.520 1.052 0.821 0.661 0.531

D53
CFA estimates
o Heywood causes:

Identification
Poor start values
Small N, 2 inds./factor
D54
CFA analysis
o Testing strategy:

1. Fit 1-factor model

2. Nested under higher-order
2
3. Compare with χ D

D55
 EHM ENR EWO EGC ETr ESM EMA EPS
1 1 1 1 1 1 1 1

Hand Number Word Gestalt Spatial Matrix Photo

Triangles
Movements Recall Order Closure Memory Analogies Series

1 1

Sequential Simultaneous
Processing Processing

EHM ENR EWO EGC ETr ESM EMA EPS

1 1 1 1 1 1 1 1

Hand Number Word Gestalt Spatial Matrix Photo

Triangles
Movements Recall Order Closure Memory Analogies Series

General

D56
CFA analysis
o Example: 4-factor model:

4 vs. 3
4 vs. 2
4 vs. 1
D57
CFA respecify
o Options:

1. Number of factors
2. Indicator-factor match
3. Error correlations
D58
CFA respecify
o Residual patterns:
Result Correlation residuals Respecification

Indicator has low High correlation residuals Switch loading of indicator

standardized loading on with indicators of another to other factor
original factor factor

Indicator has reasonably High correlation residuals Allow indicator to also

high standardized loading with indicators of another load on the other factor
on original factor factor
Allow measurement errors
to covary

D59
CFA respecify
o Wrong number of factors:

Discriminant validity

Convergent validity

D60
CFA respecify
o MIs in latent variable models:

Approach with caution

Nonsensical respecification
May not be identified
D61
 EHM ENR EWO EGC ETr ESM EMA EPS
1 1 1 1 1 1 1 1

Hand Number Word Gestalt Spatial Matrix Photo

Triangles
Movements Recall Order Closure Memory Analogies Series

1 1

Sequential Simultaneous
Processing Processing

Observations = v (v + 1)/2 = 36
Parameters = 17
dfM = 19

D62
 Exogenous variables

Direct effects on endogenous variables Variances Covariances Total

Sequential → NR Sequential → WO Seq, Sim Seq Sim 17

Simultaneous → Tr Simultaneous → SM E terms (8)
Simultaneous → MA Simultaneous → PS

D63
Example
o Amos
o EQS
o lavaan
o LISREL
o Mplus
o Stata
D64
title: principles and practice of sem (4th ed.), rex kline
two-factor model of the kabc-i, figure 9.7, table 13.1
data: file is "kabc-mplus.dat";
type is stdeviations correlation;
nobservations = 200;
variable: names are handmov numbrec wordord gesclos triangle spatmem
matanalg photser;
analysis: type is general;
model:
Sequent by handmov numbrec wordord;
Simul by gesclos triangle spatmem matanalg photser
! first indicator in each list is automatically
! specified as the reference variable
output: sampstat modindices(all, 0) residual standardized tech4;
! requests sample data matrix, residual diagnostics,
! modification indexes > 0, all standardized
! solutions (STDYX is reported), and estimated
! correlation matrix for all variables

D65
3.40 2.40 2.90 2.70 2.70 4.20 2.80 3.00
1.00
.39 1.00
.35 .67 1.00
.21 .11 .16 1.00
.32 .27 .29 .38 1.00
.40 .29 .28 .30 .47 1.00
.39 .32 .30 .31 .42 .41 1.00
.39 .29 .37 .42 .58 .51 .42 1.00

D66
CFA indicators
o Indicators:

Scale: Default ML

Likert: Other method

D67
CFA indicators
o Item distributions:

1. Binary (e.g., T / F)
2. Likert (3-6)
3. Likert (≥ 7)
D68
CFA indicators
o Estimation options:

1. Corrected ML:
a. Robust SEs
b. Santorra-Bentler
D69
CFA indicators
o Estimation options:

2. Robust WLS:
a. Item thresholds
b. Latent response variable
D70
CFA indicators
o Threshold:

Location on latent dimension

Differentiates categories
Estimated as z
D71
Example: 1 = disagree
2 = not sure
3 = agree

−1.62 1.15

D72
X1 X2 X3

EX * EX * EX *
1 2 3
1 1 1

X1* X 2* X 3*

D73
CFA indicators
o Latent response variables:

Sample polychoric
Predicted polychoric
Correlation residuals

D74
CFA indicators
o Estimation options:

3. ML + numerical integration
a. ↑ computation
b. Markov chain Monte
Carlo
D75
D76
CFA indicators
o Estimation options:

4. IRT, ICC
a. Difficulty, discrimination
b. Logit, probit link

D77
 1.0

.9

Probability of Correct
.8
.7

Response
.6 ICC
.5
difficulty
.4
tangent line
.3
.2
.1
0

−3.0 −2.0 −1.0 0 1.0 2.0 3.0

Latent Ability (θ)

D78
CFA indicators
o Estimation options:

5. Bootstrapping:
a. Very biased small N
b. Not as developed
D79
CFA indicators
o Estimation options:

6. Create parcels:
a. Homogenous item set
b. Total score
D80
 It 1 It 2 ●●● It 33 It 34 It 35 ●●● It 66 It 67 It 68 ●●● It 99

1 1 1

A B C

Pr 1 Pr 2 Pr 3 Pr 4 Pr 5 Pr 6 Pr 7 Pr 8 Pr 9
(It 1–It 11) (It 12–It 22) (It 23–It 33) (It 34–It 44) (It 45–It 55) (It 26–It 66) (It 67–It 77) (It 78–It 88) (It 89–It 99)

1 1 1

A B C

D81
Cautions about parcels
1. Assumes unidimensional

2. Ways to parcel

3. Mask multidimensionality

D82
CFA indicators
o Edwards, M. C., Wirth, R. J., Houts, C. R., & Xi, N.
(2012). Categorical data in the structural
equation modeling framework. In R. Hoyle
(Ed.), Handbook of structural equation
modeling (pp. 195–208). New York: Guilford
Press.

o Wirth, R. J., & Edwards, M. C. (2007). Item factor

analysis: Current approaches and future
directions. Psychological Methods, 12, 58–79.

D83
CFA indicators
o Bernstein, I. H., & Teng, G. (1989). Factoring items and
factoring scales are different: Spurious evidence
for multidimensionality due to item categorization.
Psychological Bulletin, 105, 467–477.

o Bandalos, D. L., & Finney, S. J. (2001). Item parceling

issues in structural equation modeling. In G. A.
Marcoulides and R. E. Schumaker (Eds.), New
developments and techniques in structural
equation modeling (pp. 269–296). Mahwah, NJ:
Erlbaum.

D84
Exploratory SEM
o CFA-EFA-SR hybrid

o Restricted + unrestricted

o EFA part is rotated

D85
 EY1 EY2 EY3 EY4 EY5 EY6
1 1 1 1 1 1
1 1
E X1 X1
Y1 Y2 Y3 Y4 Y5 Y6

E X2 1 X2 A

1 1
E X3 1 X3
C F
1 1
E X4 X4 1
DC DF

E X5 1 B
X5

E X6 1
X6

D86
Exploratory SEM
o Marsh, H. W., Morin, A. J. S., Parker, P. D., &
Kaur, G. (2014). Exploratory structural
equation modeling: Integration of the
best features of exploratory and
confirmatory factor analysis. Annual
Review of Clinical Psychology, 10, 85–
110.

D87
D88