PSYCHOMETRIKA--VOL.24, NO.

3
SEPTEMBER, 1959

THREE MULTIVARIATE MODELS:

F A C T O R A N A L Y S I S , L A T E N T S T R U C T U R E ANALYSIS, A N D
L A T E N T P R O F I L E ANALYSIS*

W. A. GIBsoNt
PERSONNEL RESEARCH BRANCH, THE ADJUTANT GENERAL'S OFFICE

The factor analysis model and La~arsfeld's latent structure scheme

for analyzing dichotomous attributes are derived to show how the latter
model avoids three knotty problems in factor analysis: communality esti-
mation, rotation, and curvilinearity. Then the latent structure model is
generalized into latent profile analysis for the study of interrelations among
quantitative measures. Four latent profile examples are presented and dis-
cussed in terms of their limitations and the problems of latent metric and
dimensionality thereby raised. The possibihty of treating higher order empiri-
cal relations in a manner paralleling their various uses in the latent structure
model is indicated.

At an early point in Multiple-Factor Analysis ([18], p. 70), Thurstone

remarks:
It would be unfortunate if some initial success with the analytical
methods to be described here should lead us to commit ourselves to them
with such force of habit as to disregard the development of entirely differ-
ent constructs that may be indicated by improvements in measurement and
by inconsistencies between theory and experiment.

This paper is an a t t e m p t to take that statement to heart.

First the derivation of the factor analysis model will be sketched, noting
three inherent conceptual and procedural problems: (i) how to estimate
communalities in the event that only shared variance is to be analyzed, (ii)
how to resolve rotational indeterminacy, and (iii) what to do with the extra
linear factors t h a t are forced to emerge when nonlinearities occur in the data.
Some recent concepts for the multidimensional analysis of quahtative d a t a - -
the latent structure model of Lazarsfeld [15]--are considered, with special
reference to their handling of the trouble spots in factor analysis. Next these
new concepts are generalized to produce an alternative way of analyzing the
interrelations among quantitative measures. This is the latent profile model.
All three of these models are discussed strictly from the point of view of
sample statistics. The problem of generalizing to a population of which the

*The latter model is anticipated in an earlier paper by Green [12].

tThe major portion of this paper was completed at the Center for Advanced Study
in the Behavioral Sciences. The opinions expressed are those of the author and are not to
be construed as reflecting official Department of the Army policy.
229
230 PSYCttOMETRIKA

sample may be considered representative is not taken up for any of the three
models.
Four examples of the latent profile model are given. Two of them are
fictitious but their form closely parallels the corresponding two empirical
examples. These examples are discussed from the viewpoint of the further
work they suggest concerning the metric and dimensionality of the latent
space. The use of higher order empirical relations for testing fit and for
further particularization of the latent profile model is briefly discussed, and
the possibility of close parallelism between developments in the latent
structure and latent profile models is pointed out. The paper concludes with
a plea for continued flexibility in the choice of multivariate models as new
and improved ones appear.

S o m e Problems in. Factor A n a l y s i s

The fundamental postulate of factor analysis ([18], p. 63) is expressed in
the simple linear equation
(1) Z , = ailg,1 + aj2Z,~ + . . . + a~Z,~ .
Z , is the standard score of individual i on test j. Z,.1 , Z~2, ..- , Z~ are the
standard scores of individual i on a hypothesized set of q statistically in-
dependent traits or factors. (This does not preclude subsequent conversion
to correlated factors in any given analysis. The algebra of correlated factors
will not be introduced here, however, for that would only complicate the
discussion without changing the arguments.) The a's in (1) are a set of
weights descriptive of test j and invariant for individuals.
Straightforward summational algebra and the independence of factors
lead directly from (1) to the basic equation of factor analysis ([18], p. 78):
(2) r,-~ = a~lakl + a~2ak~ + . . . + a ~ a ~ .
Thus r;k , the correlation between tests j and k, is expressed as a simple
bilinear function of the a's for those two tests. These a's, also known as
factor loadings, are interpretable as correlations between tests and factors.
The essential task in the factor analysis of a battery of s tests is to solve
for the a's in the system of s(s -- 1)/2 bilinear equations resulting from (2).
The number of factor loadings is sq, q for each of the s tests. These loadings
can be obtained by a wide variety of techniques that have been developed
over the years. Most of these methods attempt to account for the inter-
correlations in terms of a minimum number of factors. Perfect accounting
for the intercorrelations by the factors is seldom demanded because of
sampling error and the frank expectation of at least minor disagreements
between model and data. Some nonvanishing "residual" correlations are
permitted, so long as they are smM1 and show no systematic pattern.
One troublesome feature of the factor model is the problem of how to
 w. A. GmsoN 231

deal with those elements in the correlation matrix that have repeated sub-
scripts--the diagonal ri/s. To take these as perfect self-correlations of unity
would amount to trying to analyze all of the variance of every test, including
the unreliable part. To insert the test reliabilities into the diagonal cells would
imply an interest in analyzing all of the "true" or repeatable variance of the
tests, including that specific to each test and unrelated to other tests in the
battery. More commonly preferred among factor analysts is to attempt to
analyze only that part of a test's variance---its communality--that is shared
with other tests in the battery. Communalities could be defined alternatively
as those portions of the self-correlations accounted for by the factors that
suffice to account for the between-test correlations. In any event, the com-
munalities are not empirically given and yet are needed at the start for maxi-
mum efficiency of solution. In principle the communalities could be determined
by certain operations (cf. [18], pp. 294-307) applied to the empirically given
side entries in the correlation matrix, but these operations are usually so
time consuming that a successive approximations approach is often sub-
stituted. Rough communality estimates are used to obtain an initial factorial
solution, which in turn provides improved estimates for a second cycle of
the same kind, and so on until the communalities are snfficiently "stabilized."
It fortunately happens that with large test batteries little or no iterating
of this kind is needed, but for small batteries several cycles may be required
before the communality problem is adequately resolved.
A second and more important problem in factor analysis is the inherent
partial indeterminacy of the a's that is known as the rotational problem. It
is most easily understood in terms of q-dimensional geometry. The a's for
any test j may be thought of as the projections of a point j on a set of co-
ordinate axes in q-space. The table of factor loadings for the s tests then
defines a configuration of s points in terms of their projections on a q-di-
mensional reference frame. But the equations of factor analysis, by themselves,
do not indicate which position of the reference frame, among an infinite
number of possibilities in the same q-space, is to be preferred. Only the origin
of the coordinate system is fixed, so that the reference frame can be rotated
freely from any position to any other without distortion of the configuration
of points defined by it. Naturally the a's change with such rotations, but
always in such a way as to preserve the spatial interrelations among the
points they define. Many ways of resolving this rotational indeterminacy
have been proposed. Probably the most notable is the simple structure
principle ([18], pp. 181-193), which strives to simplify the factorial structure
of the tests by maximizing the number of near-zero factor loadings. Many
of these proposals (especially those centering around the simple structure
principle) involve heavy computing loads, relatively rare geometric intuition,
or both; many are debated or debatable; all are in the nature of afterthoughts
that are not built into the equations defining the model.
232 PSYCHOMETRIKA

Another perplexing problem in the factor analysis model is the paradox

of difficulty factors [2, 4, 13, 20]. If factor analysis is applied to a battery of
tests varying widely in difficulty but quite obviously measuring but one
underlying trait, the result is not one but several " f a c t o r s " - - o n e for each
level of difficulty. This is generally attributed to curvilinear relations among
tests markedly different in difficulty, such curvilinearity being forced by the
differential skewness of the score distributions. Note, however, that it is
only the relations between tests and factors, as indicated by (1), that the
factor model explicitly restricts to linear form.
The record of empirical fruitfulness (or lack thereof) of factor analysis
is not at issue here. Nor are misapplications of it pertinent to this discussion.
The other two models to be discussed here are meant to be put to much the
same use, and they may very well suffer the same kind of misuse.

Some N e w Conzepts: Latent Structure A n a l y s i s

Only one variety of latent structure analysis--that known as the dis-

crete class model--is discussed here.
Latent structure analysis [15] is Lazarsfeld's technique for analyzing
the interrelations among dichotomous attributes, such as the item responses
on a survey questionnaire. It is based on linear recruitment equations of the
following kind:

n =nl +n~ + ... +n~,

n~ = nlp~i + n2p2i + "'" + n~p~ ,

(3)
njk = n~p, ik + n2p2ik + "'" + n ~ p ~ ,

nikz = n~plik~ + n~p2~k~ + . ' " + n~p~ikz , etc.

The quantities on the left are empirically given or manifest data. They
indicate the number of people in the entire sample, n, the number endorsing
a single item, n, , the number endorsing any pair of items, n,~ , and so on.
The quantities on the right are the underlying or latent parameters of the
model. The number of terms on the right is q, the number of mutually exclu-
sive and exhaustive subgroups (latent classes) into which the analysis will
divide the total sample. The number of people in latent class 1 is n~ , and
so on. The latent probability Pli is the proportion of the members of latent
class 1 who endorse item j, Pl~'~ is the proportion of class 1 members who
endorse both items j and k, and so on. Equations (3) merely show how the
man~fest joint endorsements are recruited from the latent classes. It may
be noted, in passing, that equations (3), being a set of recruitment equations,
are intrinsically linear, while the initial equation of factor analysis is linear
only because it was made so.
 w . A. ~ m S O ~ 233

The manifest-latent distinction that is so prominent in latent structure

theory is of course central in factor anatysis as well. There the tests and
their intercorrelations are manifest, and the factors and the loadings thereon
are latent. The same distinction also appears in the latent profile model to
be discussed later.
The preceding recrttitment equations are equally valid for any method
of classification and for any number of latent classes. The next step is to
invoke a pertinent basis for classification. This is the core of the latent struc-
ture model. I t is quite relevant to require that each latent class exhibit
homogeneity with respect to any underlying dimensions that m a y be re-
sponsible for the manifest interrelations. Perfect homogeneity is not crucial,
so long as deviations from the class norm are random. Such random deviations
are of course uncorrelated within the class. Thus it is sufficient to require
that each latent class be homogeneous enough, with respect to any and all
such latent dimensions, so that all item responses w i t h i n t h e c l a s s are independ-
ent in the coin-tossing sense. This intraclass independence is expressed in the
following equations.

(4) plik = pljpl~ , P2i~ = p ~ p 2 ~ , " ' " , P~i~ = p~ip~k ,

p~k~ = P~PI~Plt , P2~ = p2~p2kp~ , " ' " , p~ = p~p~p~ , etc.

The substitution of (4) into (3) yields the basic equations of latent
structure analysis ([15], p. 385).

n = n~ + n2 + ... + n, ,

n i = n~p~i "4- n2p2~ ~ - " " + n~p~i ,

(5)
n~k = n l p l i p l ~ -Jr" n2p2ip2k + "" -{- n ~ p ~ i p . ~ ,

n~kz = n l p l ~ p l ~ p l l A - n2p2ip2~p2~ -I- " . -4- n ~ p ~ i p ~ k p ~ , etc.

Thus all of the manifest joint frequencies are accounted for in terms of
(q A - s q ) latent parameters, q class sizes and q latent probabilities (Pl~ ,
p~ , -. • , p,~) for each of the s items. The successive levels of manifest fre-
quencies (n, n~ , ni~ , etc.) number, respectively, 1, s, s ( s - - 1 ) / 2 , etc., the
coefficients in the binomial expansion (a -t- b)L These add up to 2", the
number of equations relating manifest to latent data in this model.
The task here, as in factor analysis, is to solve the basic equations for
the unknown latent parameters. Several latent structure solutions already
exist. The most recent of these [1, 7] avoid the use of any joint frequencies
with repeated subscripts (n~.~ , n~,.~ , n;~'i , n;~k, , etc.). In latent structure
analysis these are treated as analogous to the communalities of factor analysis,
in not being manifest. To interpret them as equivalent to the corresponding
234 PSYCH01~IETRIKA.

lower order joint frequencies without repeated subscripts (i.e., n;; = n; ,

n ~ = n~k, n~i~ = n~, etc.) would be analogous to the use of unit self-corre-
lations in factor analysis. Instead, the latent structure model usually treats
these elements with repeated subscripts as stemming from, rather than
leading to, the latent parameters of the model that suffices to account for
the manifest data without repeated subscripts.
The Anderson latent structure solution and an earlier one by Green [11]
eliminate the latent structure analogue of the rotational problem through
the use of manifest data with more than two subscripts, such as n;~z • (A
cursory investigation indicates that it might be self-contradictory for the
factor model to make use of manifest interrelations among more than two
variables at a time. This is because at least some of the joint distributions
among factor scores would have to exhibit asymmetries that could easily
destroy the basic linear postulate of factor analysis.)
These higher order data select, from the infinite number of rotational
solutions accounting for lower order manifest frequencies equally well, the
o~e that fits themselves best. Another early latent structure solution [5, 6, or 8]
effects a partial (often severe) reduction of rotational indete~linacy without
the expense of obtaining higher order data. This is accomplished by capital-
izing on the simple fact that the latent parameters, being probabilities, can
be neither negative nor greater than unity.
Earlier it was indicated that the artifact of difficulty factors arises in
factor analysis when curvilinearities are present. Note that the derivation
of the latent structure model embodies no restriction on curvilinear relations,
either among the manifest attributes or between them and the latent
dimensions.
The latent structure model has already shown promise in empirical
research [ef. 14, 17]. Only its restriction to the analysis of dichotomous
attributes prevents it from being a feasible alternative to the factor analysis
of quantitative variables. The latent profile model to be taken up next is the
generalization of latent structure analysis to the case of quantitative manifest
variables [cf. 12].
Some Linear Recruitment Equations ]or Quantitative Manliest Variables
A reasonable question to ask is whether the natural linearity of recruit-
ment equations could not provide a basis for analyzing interrelations among
manifest variables that are quantitative rather than qualitative. This section
will display such a system of recruitment equations.
Suppose there is available a set of s quantitative measures, such as
test scores, on a sample of n people. On some basis let every sample member
be assigned to one, and only one, of q subgroups. Then the size, sums of
scores, and sums of score products for the entire sample are related to corre-
sponding subgroup statistics in the following simple way.
 W. A. GIBSOI~ 235

n =nl +n~ + ... +n~,

+ + ... + etc.

All summations in (6) are over individuals. The summations on the left
are over the entire sample, while those on the right are over the members of
the various subgroups. X , is the, score of individual i on test j, and it m a y be
in raw, deviational, standard, or a n y other derived units. The same is true
of X ~ , X ~ , etc.
I t is convenient to designate the various summations in (6) b y the
letter m with appropriate subscripts. Thus mi and m;k represent, respectively,
the sum of scores on test j and the sum of products of scores on tests j ahd J k,
each sum being for the entire sample, ml~ and mxi~ , on the other hand,
stand for the same things in subgroup 1 only, and so on. Equations (6)!then
become
n=nl + n ~ + "'" + n ~ ,

(7) mi = mli + m2j + " " + mqi ,

m i k = m t ~ + m2;k + " ' " + m~ik ,

mik~ = m~j~ + m2ik~ + "'" + mqikt , etc.
T h e m's in (7) will be referred to as product moment.s of various orders.
T h e product m o m e n t s in the third line, involving two tests, will be said to
be of second order, while those for three tests are of third order, and so on.
For consistency, the m's involving only one test j will be called first-order
product, moments, and n, the sample size, could be called the zero-order
product m o m e n t for the sample. T h e m's on the left in (7) will be called
sample product moments, while those on the right are subgroup product
moments.
I t should be stressed t h a t (6) and (7) in no w a y restrict either the em-
pirical d a t a or the method of classification into subgroups. I t will be the
burden of the next two sections to establish a basis for grouping t h a t will
convert these recruitment equations into a m a t h e m a t i c a l model.

The Fundamental Theorem of Latent Profile A n a l y s i s

Consider now a two-dimensional joint distribution in which the score
of every person in the sample on test k is plotted against his score on test j.
236 PSYCHOMETRIKA

Such a scatterplot is pictured in Figure 1. The unit of measurement for both

tests is entirely arbitrary here. The ellipse in Figure 1 indicates only one of
the possible shapes that the total configuration of points could have. The
circle represents a subgroup t, of size n , , within which the correlation between

Xk Xtj

dikll II
X t~
dij

0 -- Xi

FIGURE 1
A Hypothetical Scatter Diagram

tests j and k is zero. The two lines labeled X , and Xtk in Figure 1 indicate
the means of subgroup t on tests j and k. They intersect in what is called the
centroid of the points comprising subgroup t. The point corresponding to
individual i , a member of subgroup t, is shown with the coordinates
( X , , X~k), his scores on the two tests. The two distances, d , and d~k , are
the deviations of individual i from the j and k means of his subgroup. A
property of such deviations is that their sum, over all members of the sub-
group, is zero.
 W. A. GIBSON 237

The j and k scores of individual i may be expressed in terms of his sub-

group means and his deviations from those means, as follows:
(8) X. = X. + d. ,
(9) X , k = X , ~ + d,k •

Then m~ , the sum of j scores for subgroup t, is given by

(10) ms; = k X .

= k ( X , ; + d.)

= ~,X,,+ 5~d,

= ntXti •

The first term in the third line simplifies because X , is the same for every
member of the subgroup. The last term in the third line vanishes because it is
a sum of deviation scores. Thus the j scores for any subgroup t contribute
to the sum of all j scores as il the members of the subgroup were concentrated
at their mean for test j. This result (by no means a new one) will next be
generalized, in an appropriate way, to the second-order product moment,
m , ~ , for subgroup t.
B y definition and with the help of (8) and (9), *n,~ becomes

(11) m,i~ = z ~ X , X i ~

= k (X,i + d,~)(X,~ + d,k)

= k (X.X,~ + X , i d,k + X,~ d . + d . d,~)

= k<<
= n~XtiXt~ .

T h e first term in the fourth line of (11) simplifies because X , and X,k are
the same for all members of the subgroup. The second and third terms in t h a t
line vanish because they contain sums of deviations. The last term vanishes
because it is the numerator of the formula for the correlation, within the
subgroup, between tests j and k. The subgroup was earlier defined as having
t h a t correlation equal to zero. Thus the second-order product moment,
m , ~ , is the same as it would be if all members of subgroup t were concentrated
a t their centroid. This holds for any subgroup within which tests j and k are
uncorrelated.
A further distinction in terminology needs to be made here. The quantity
~38 I~SYCHOMETRIKA

m,i~ is a sum of products of horizontal and vertical distances from the origin
in Figure 1. It will therefore be called a product moment about the origin.
Another kind of product moment in (11) is the last term in line 4, the sum
of products of horizontM and vertical distances of a set of points from their
own centroid. Such a quantity, is appropriately called a product moment
about the centroid of the set of points that is involved. The geometric interpre-
tation of scores and deviations as distances makes it clear that the point of
reference (origin, centroid, or some other point) of a product moment must
Mways be specified in some way. Naturally this holds for all orders of product
moments. Up to now, with four exceptions, all product moments have had
the origin as their point of reference. The four exceptions are the last term
in line 3 of (10) and the summations in the last three terms of line 4 in (11),
which have the centroid of subgroup t as their point of reference.
Theresults of (10) and (11) can be generalized to higher order product
moments by imposing additional restrictions on subgroup t. Not only must
that subgroup be defined as having all pairs of tests j,/c, and l uncorrelated
within it, but, let it also have, for those three tests, a vanishing third-order
product moment .about its centroid. With these restrictions its third-order
product moment about the origin, mr;k, , becomes
(12) m,~ = n~X,X,~X, .
This result is, in form and mode of development, a third-order analogue of
the final step in the previous two equations. The fourth-order equivalent is
obtained by anMogous higher order restrictions, and so on.
Since in all of this discussion the origin could have been placed at any
point 0, it now becomes possible to state the fundamental theorem of latent
profile analysis.
The g-order product moment, about any point 0, of n, points having
zero product moments of order g and less about their centroid, is
equal to the g-order product moment, about the point 0, of n~ points
placed at that centroid.
The Basic Equations o] Latent Profile Analysis
The foregoing theorem provides a basis for grouping in the recruitment
equations introduced previously. The close analogy with latent structure
analysis will be obvious. Each subgroup or latent class should be homogeneous
in whatever underlying dimensions are necessary to account for the observed
interrelations. The homogeneity need not be complete, so long as deviations
from the class averages are random, i.e., independent.
In the statistics of dichotomous attributes (as employed in coin-tossing
experiments, for example)., the notion of independence has usuMly applied
to all orders of joint occurrence, and not just to pairs of events. This is the
case in latent structure analysis, where within-class independence is defined
 w . A. Q m s o ~ 239

as pertaining not only to all pairs of items but also to all triplets, all quad-
ruplets, and so on. (It is easily shown by example that higher order independ-
ence among dichotomous attributes is not a mere logical consequence of
uneorrelatedness between all pairs of such attributes.)
The concept of uncorretatedness among quantitative measures, on the
other hand, ha.s more often been restricted, at least among psychometricians,
to pairs of such measures. (A statistically oriented prepublieation reviewer
has pointed out that the two kinds of independence being discussed here are
known to statisticians as pair-wise and mutual independence.) This is not an
intrinsic or logical difference between qualitative and quantitative statistics.
It is rather only a historical accident that the question of higher order in-
dependence among quantitative measures has less often arisen in psycho-
metrics. That question arises here, for it turns out that the proper definition
of such independence is crucial for this model.
In the previous section the within-class uncorrelatedness between pairs
of tests was shown to be synonymous with vanishing second-order product
moments about tile eentroid of the class. This is because such product
moments are the numerators of the formulas for the corresponding correla-
tions. Purely by analogy, higher order within-class independence may be
equated with the vanishing of higher order product moments about the
centroid of the class. (The failure of such.product moments to vanish would
allow, for example, a positive correlation between tests j and h among class
members with high scores on test l, accompanied by a compensating negative
correlation between the same two tests among class members having low
scores on test 1. This could happen in spite of zero correlations between all
pairs of the three tests within the class as a whole. If correlational patterns
can differ within subdivisions of a class, then the class is not homogeneous
even from a commonsense point of view.) Therefore let the within-class
independence of the present model be defined as applying to all orders of
interrelations, and as expressing itself in the vanishing of product moments
of all orders about the eentroid of the class. Then the fundamental theorem
applies with full force to the product moments of each class, so that the results
of equations (10), (11), and (12) can be used to transform (7) into the basic
equations of latent profile analysis:

n=nl+n2+ ... +n~,

(13) m~ = n~X~ + n2X21 + .." + n , X , i ,

mi~ = n ~ X ~ X ~ + n~X~iX~ + . . . + n ~ X , i X ~ ,
m i k , = n~X,,.X,~Xxz + n2X2~X2kX2l + "'" + n~X~iX~kX~ , etc.

Thus for s tests the 2 ~manifest product moments (including n) are accounted
for in terms of (q + sq) latent parameters--the q latent class sizes and the
240 PSYCHOMETRIKA

q class averages for each of the s tests. Each latent class is therefore character-
ized by its size and its profile of s test averages, its latent profile.
The latent profile equations and those of latent structure analysis turn
out to be identical in form, as can be seen from a comparison of (5) and (13).
This means that all Mgebraic latent structure solutions [1, 7, 11] are directly
applicable to the latent profile equations. Hence the latent profile equations
have a solution that can be obtained without the involvement of eommunality
analogues (m~i , m i l k , m l i i , etc.), and that is, in general, rotationally unique.
(A conversation with Robert P. Abelson at Yale University has clarified the
fact that, when deviational scores are used, the m~ are the between-groups
variances.) Nor do the latent profile equations restrict the occurrence of
curvilinear relations among tests or between tests and underlying dimensions.
Thus the dilemma of difficulty factors is avoided in this model.

Two Special Cases of Latent Profile A n a l y s i s

In the development of the latent profile equations the score units were
entirely arbitrary. There are, however, two kinds of test scores that deserve
special attention. Consider first the case where the manifest variables are, as
in latent structure analysis, dichotomous attributes. Let the presence and
absence of each such attribute be designlated by scores of one and zero,
respectively. In this case the manifest latent profile m's become identicM
with the manifest latent structure n's, and the class averages of latent profile
analysis become the latent probabilities of latent structure analysis. The
latent class sizes mean the same thing in both models. Thus the latent
structure model is interpretable as the speciM case of latent profile analysis
in which the manifest variables are dichotomous.
A second special result is obtained by using standard scores and by
dividing the latent profile equations through by n, the number of people in
the sample. The latent profile equations then assume their standard form:

1 = Pl + P 2 + "'" + P ~ ,
0 = p l Z l l -4- p~Z2~ 4- " . + p,Z~ ,
(14)
r~k = piZliZlk + p2Z2iZ2~ + "'" + PQZaiZq~ ,
r~k, = p,ZliZIkZ1, + p~Z~Z:~Z2z + . . . + poZqiZ~Z~, , etc.

The p's are the proportionate class sizes, and of course their sum is unity.
The Z's are the average standard scores of classes on tests. Their weighted
sum for any test (in the second line) vanishes because it is the mean of all
standard scores on that test. The r~-, , being average products of pairs of
standard scores, are the same correlations for which factor analysis attempts
to account. The r;~.., are, analogously, average triple products, and so on.
 w. A. Gmso~ 241

The latent profile equations in standard form have the advantage of dealing
with magnitudes that are independent of sample size and of arbitrary score
units. It is in standard form that the equations will be applied to the examples
in the next four sections.

Latent Profile Example I: A Fictitious Two-Class Case

The fictitious manifest data in Tables 1 and 2 will serve as a first latent
profile example. Imagine the data as resulting from the administration of
four alternate forms of the same test (such as arithmetic) to a sample of, let
us say, a hundred people. Suppose, further, that all six intercorrelations turn
out to be .50, so that every two-dimensional scatter diagram, with scores
plotted in standard units, has the appearance of an ellipse twice as long as
wide, centered on the origin, and tilted at an angle of 45 degrees. There are
four three-dimensional scatter diagrams. Each is approximately egg-shaped,
and, being symmetric about the origin, yields a third-order manifest product
moment of zero.
Tables 1 and 2 display the necessary manifest data in a convenient way.
The upper left entry in Table 1 is the first term in the first line of (14), the
latent profile equations in standard form. The other entries in row and
column 0 of Table 1 are the means of standard scores fdr each of the four
tests--the left-hand term in the second line of (14). The remaining cells of
Table 1 contain the test intercorrelations--the manifest data in the third
line of (14).
Table 2 summarizes the manifest data of first, second, and third order.
The upper left entry is the sum of the four means of standard scores. Each
of the other entries in row or column 0 is the sum of the four correlations
(including r;,) involving the associated test. Every other cell in Table 2
contains the sum of the four third-order mar~est product moments (including
r~;~ and r;k,,) for the corresponding pair of tests.
For convenience of exposition in both fictitious examples in this paper,
all elements with repeated subscripts (such as r i i , r i i , , and r , 3 are treated
as known. Their values are, in fact, easily inferred from the simple form of
the manifest data, but this would not be true generally, even for all sets of
fictitious data. In this first example, all r , are .50, all r,k are zero, and all
r,;~- are zero.
Tables 1 and 2 have been labeled R and R, respectively. This is con-
venient notation for any such display of manifest data, and it will be used
in all examples. In any latent profile solution, a distinction must be made
between the given and the fitted R and R~, the latter pair of tables indicating
what the former should be in order to be completely accounted for by the
solution. In both fictitious examples in this paper, the given and the fitted
manifest data, the latter computed from the solution by means of (14),
are identical and hence need not be compared. In the two empirical examples,
 TABLE i TABLE 2 TABLE 3

R for a Fictitious Two-Class Case R I for a Fictitious Two-Class Case Latent Profile Solution for
a Fictitious Two-Class Case

Test Test Number Test Test Number ITest Latent Class

No. O i 2 3 No. 0 i 2 3 4
O i .O0 .00 .O0 .00 .00 0 .00 2.00 2.00 2.00 2.00 -.71 .71
1 .co .5o .5o .5o .9o I 2.00 .00 .0~ .00 .00 Class -.71 .71
2 .oo .50 .90 .5o .5o 2 2.00 .00 .00 .00 .00 Means - .71 .71

3 .oo .5o .90 .9o .90 3 2.00 .00 .00 .00 .O0 -.71 .71
4 .oo .9o .9o .5o .90 4 2.00 .00 .00 .00 .00 Class Sizes .90 .90

co
TABLE 4 TABLE 9

Given Correlations for Nine Readln~ Tests Approximate Latent Profile ©

Solution for Nine Readin@ Tests

Test Test Number Test Latent Class

No. 0 .........1 2 ....3 4 9 6 7 ~ ~ ....... No. I II
0 1.00 .CO .CO .00 .CO .CO .00 .00 .00 .00 i -.80 .80
l .oo .... 72 .~i .28 .52 .71 .68 .~l .68 2 -.81 .81
2 .00 .72 .... 34 .36 .53 .71 .68 .52 .68
3 .00 .41 .34 .... 16 .34 .43 .~2 .28 .41 4 -.41 .~i
.00 .28 .36 .16 .... 30 .36 .35 .29 .36 Class 9 -.68 .68
9 .oo .52 .93 .34 .3o . . . . 64 .59 .49 .95 Means 6 -.90 .90
6 .00 .71 .71 .43 .36 .64 . . . . 76 .97 .76 ? -.8~ .89
7 .00 .68 .68 .~ .35 .95 .76 .... 59 .68 8 -.66 .66
8 .00 .51 .52 .28 .29 .49 -97 .59 .... 58 9 -.84 .84
9 .00 .68 .68 .41 .36 .55 .?6 .68 .98 ...
Class Sizes .90 .50
 w. A. GIBSON 243

however, the comparison will be made when possible in order to appraise the
adequacy of the solution.
The latent profile solution for the present example, obtained by applying
the algebra of the latent structure solution of Green Ill], is shown in Table 3.
Each of the two latent classes is defined in terms of its relative size and its
latent profile---the complete set of average test scores for its members.
Apparently Class I consists of those who are poor at arithmetic, while Class
II contains the good arithmetic students.
A fruitful way to visualize this latent profile solution is in terms of the
regressions of the tests on the latent continuum of arithmetic ability: Such
a graph of mean test score, Y, against position along the latent continuum,
X, is shown in Figure 2. Here the regressions of all four tests on the latent

Y
1.00

Tests
I-4

Glossl!
I" IX
-I.00 l.O0
Clas! II

Tests/
I-4
-!.00
F~GURE 2
Regression of Tests on Latent Continuum for A Fictitious Two-Class Case

continuum are identical. In Figure 2 both Y and X are expressed in standard

units, so that the slope of the regression line is also the correlation between
Y and X. This is a correlation between test and "factor," and for the linear
regressions of the present example, these correlations turn out to be exactly
244 PSYCHOMETRIKA

the same (X/2/2) as the factor loadings that would result from a factor
analysis of the correlations in Table 1. This simple correspondence between
the two models will vanish, however, as soon as any of the regressions become
nonlinear.

Latent Profile Example II: An Empirical Two-Class Case

As a second latent profile example, consider the given R in Table 4.
That table is a simple modification of a table of intercorrelations among
nine reading tests previously reported by Davis [3]. The only modification
was to border Davis' table with the 0 column and row. The intercorrelations
were based on 421 cases.
A letter from Davis has indicated that the raw scores that would be
needed for the computation of R1 are no longer available. In the absence of
higher order data which would provide a unique solution, it was necessary
here to adapt some factoring and rotational procedures that were involved
in an early approximate latent structure solution [5, 6, or 8] in order to obtain
an approximate latent profile solution. For this purpose a factorization of
the Davis data by Thurstone [19] was used. Thurstone's analysis indicated
that one factor was sufficient to account for the data, all but three of the
discrepancies between given and fitted correlations being less than .04, and
the largest being .07. It is with exactly these same discrepancies that the
present latent profile solution accounts for the intercorrelations.
An approximate latent profile solution for the Davis data is shown in
Table 5. This solution is obtainable by resolving the rotational problem
with any one of the following three assumptions:

(i) that the two latent classes are equal in size;

(ii) that the two latent profiles are identical except for reversed algebraic
sign; or
(iii) that the given R~ , if available, would be like that of Example I in
having nonzero entries only in its 0 column and row.

The solution in Table 5 generates, by means of (14), a fitted R~ having the

form indicated in assumption 3.

The regressions for this solution are pictured in Figure 3. Again both
axes are in standard units, so that the slopes of the regressions are identical
with the factor loadings reported by Thurstone. Although the present latent
profile solution is not rotationally unique, it can fairly readily be sho~n to
possess an important kind of invariauce, namely, that the slopes of the re-
gressions, when both axes are in standard units, remain constant regardless o]
how the rotational indeterminacy is resolved. This is but one illustration of
the fact that the factor analysis and latent profile models are mutually
complementary when the assumptions of both models are not violated.
 W. A. GIBSON 245

Y
1.00 Tests

ClQss 1

........ X

-I.00
FzO~IR~ 3
Regressions of Tests on Latent Continuum for Nine Reading Tests

Latent Profile Example III: A Fictitious Three-Class Case

The fictitious R and R1 in Tables 6 and 7 w i l l provide a first illustration
of how latent profile analysis handles the problem of difficulty factors. Imagine
tests 1 and 2 as being two easy vocabulary tests, tests 4 and 5 as two hard
vocabulary tests, and 3 as a vocabulary test of intermediate difficulty.
Again assume the data are based upon a hundred cases.
Before proceeding to the latent profile analysis of this fictitious data, it
will be instructive to examine the results of a factor analysis of the correlations
in Table 6. The simple structure factor analytic solution with correlated
factors is given in Table 8. The entries in that table are the correlations
between the five tests and the two factors, A and B. The correlation between
the two factors, rA8 , is .33. If the usual rules for interpreting factors were
followed unquestioningly here, the conclusion would be that the two factors
are knowledge of easy words, A, and knowledge of hard words, B, and that
the two abilities are relatively independent. This is absurd.
The unique, perfectly fitting latent profile solution for this example,
again obtained from R and R1 by the same algebra as is used in the latent
246 PSYCHOMETRIKA

TA~I~ 6 •AB~ 8

R for a FictitiouS Three-CLass C~se Simple Structure Factor Analysis

Solution for Correlations in ~ b l e 6

Test Test ~hmloe r Test Factors

No. o 1 ~ ............
3 ~ ~5 ]~o. A ......B
0 1.00 .00 .00 .00 .00 .00 1 .8~ .00
1 .00 .7~ -73. .~0 .~3. .~3. 2 .8~ .00
.00 -75 -73. .90 .25 .23. 3 .~z .~Z
3 .oo .~o .9o .5o .5o .5o 4 .00 .82
.oo .z3. .~3. .~0 .73' .73 3. .00 .82
5 .oo .2~ .~3. .5o .T~ .73 r~ = .33

TASL~ 7 TABL~ 9

R 1 for a Fictitious Three-Class Case Latent Profile Solution for

a FietitiousThree-Claes Case

Tes~ Test Number Test . Iatent Class

~o__. ..... 0 z _e > 4 ~o. z ' '~i ' in
o .~ 2.5o 2.~o 2.~ z.~ z.~o
1 2.5o -2.~o -z.5o -1.23. .oo .oo 2 -z.9o .~o .9o
CIMe
2 ~.5o -2.5o -2.5o -1.23. .oo .oo Means 3 -l.O0 .00 l.O0
3 z.So -1.23. -1.e5 .co z.~ z.25 4 - .50 -.50 1.50
2.5o .co .oo z.z3. 2.~o 2.5o 9 - .9o -.~o z.5o
3. z.5o .oo .oo 1,z3. 2.~o 2.~o Class Sizes .z5 .70 .23.

structure solution of Green [11], is shown in Table 9. Figure 4 shows, in

standard units, the regressions implied by Table 9 and by the assumption of
equal spacing of the latent classes along the single latent continuum of
vocabulary knowledge.
The contour of the various regressions in Figure 4 is exactly what would
be expected on the basis of the relative difficulty of the tests. The easy tests
(1 and 2) discriminate only at the lower eIld of the latent continuum. The
hard tests (4 and 5) differentiate only at the upper end. Test 3, of medium
difficulty, discriminates throughout the range.

Latent Profile Example IV: An Empirical Three-Class Case

The data in Table 10 will provide a final latent profile example. Aside
from its 0 column and row, that table is merely a rounded version of a table
of subtest intercorrelations reported by Ferguson ([44 p. 328) to illustrate
the occurrence of difficulty factors. Ferguson took single items from a Moray-
House verbal intelligence test and combined them into six subtests that were
reasonably homogeneous in content but that increased in difficulty from
subtest 1 to subtest 6. The correlations were based on a saznple of 108 children,
age 11.
 %V. A. GIBSON 247

Y
Tests

1.00"

1,2
Closs I
{X:-V-f}
1I i I I X
-LO0 1.00 I
Tests CIQss IIl
4,5 - - (x=+~l

3/ / -I.00~

1,2
/ A.
IX=O)
Fmu~E 4
Regressions of Tests on Latent Continuum for A Fictitious Three-Class Case

The simple structure correlated factor so]ution for the Ferg~ison example
is shown in Table 12. That solution was obtained by a rotation of the factori-
zation reported by Ferguson ([4], p. 328). Here again it would be absurd to
conclude, according to the usual rules, that the data must be thought of in
teiTns of two relatively independent factors--high-level and low-level verbal
intelligence.
A letter from Ferguson has indicated that the raw scores are no longer
available for the computation of higher order manifest product moments.
It therefore was necessary to approximate the latent profile solution by
means of procedures similar to those used in the Davis example. For this
purpose Fergllson's factorization of his corre|ations was used. His two factors
accounted for the correlations with discrepancies not exceeding .03, and the
present latent profile solution fits the correlations in eactly the same way.
The rotational indeterminacy here was resolved in two stages. The
first step was to rotate Ferguson's factorization, with attention being given
only to subtests 1 and 6, into maxhnum correspondence with. the initial
factorization for the first and last tests in Example Ill. The latter faetori-
zation was a part of the latent profile solution for that example. In Example
III the unique solution was obtained by applying, to the initial factorization
248 PSYCH0:METRIKA

TABL~ lO

Given C~rrelations f o r S~x S i ~ e S t r u c t u r e Femtor a n a ~ s l ~

Verbal l n ~ e l l i g e n c e Sub-Tests Solution f o r C o r r e l a t i o n s ~n Table 10

Test Test Number Test Factors

No. o ~ a ~ ~ ~ 6 No, A B
0 1.00 .00 .00 .00 .00 .00 .00 1 .86 .00
Z .00 .... 86 .8.1. .SZ .6Z -)7 2 .?~ .~9
2 .oo .86 .... 8o .8~ .6~ .~7
.co .sz .8o . . . . 87 .8o .67 4 .5? .~.2
.oo .SZ .8~ .87 .... 80 .68 .~ .~o .6~
.oo .6~ .68 .8o .8o .... 78 6 .oo .8z
6 .oo .~7 -~7 .6? £8 .78 ... r~m = .~

TABLE 1 / m/BL~ 13

F~tted ~1 f o r Six Verbal Y m t e l l ~ e n c e Sub-Tests Approxim~te Imtent Profile Solution

for' Six Verbal ~J~telligence Sub-Tests

TeBt Test l~mber TeBt La%ent Cl~ss

too. o z 2 3 ~ _ ~ 6 .~ No. _ I II III
o .oo ~.3z ~.~9 ~.8o ~.~, 3.17
~.~7 l -1.6~ .~l .62
]. ~.3]. - ~ . ~ -3.95 ..a.98 -3.o7 -z.7~ -.2~, 2 -z.96 .36 .8~
2 ~,.~9 -5.93 -3.57 -2.3o -.2.39 -3..o~ .~,,~ C~us 3 -L38 .07 z.2~
Meau6
~,.80 -2.98 -2.30 - .SO -z.o8 .3~ z.79 -z.~l .09 z.23
~. ~.8~ -~.07 -2.~9 -1.08 -l.16 .26 LTO 5 -i.o9 -.20 l.b,4
.5 4.~7 -1.74 -Z.O2 .~3 .26 i.~2 2.7.1. 6 - .60 -.h8 1.56

6 ~.?? - .24 .~5 z.?.5 z.7o 2.7z ~.~7 Class Sizes •29 ,9o .25

of R, a rotation completely specified by the higher order data in R1 . The

second rotational stage in the present approximate solution was to imitate
the Example I I I solution by using exactly the same rotation. The conse-
quences of this rotational solution are the following.
(i) The relative class sizes are the same as in Example III.
(ii) The regressions of all six subtests on tile latent continuum are
ascending.
(iii) Assuming equal spacing of the classes along the latent continuum,
the regressions of subtests 1 and 6 have curvatures that are approximate]y
equal but opposite in direction.
(iv) The form of the fitted R1 is similar to that of the given and fitted
RI of Example III.
The resulting approximate latent profile solution is given in Table 13, and
the regressions, again in standard units, and assuming equal spacing of the
three classes along the latent continuum, are shown in Figure 5. The progres-
sion of curvatures from the easiest to the hardest subtest is just what it
ought to be. This progression was found to be quite invariant over a wide
range of alternative approximate solutions. Several such solutions were
 W. A. GIBSON 249

Y
Tesls
.6

1.00"

Class l
(x=~,]TI
I I !
'X
-I.(X) 1.00 T
Class III
Tests (x= +/'~)

-'°°t
Cla!s 11
x=o)
FIGURB 5
Regressions of Subtests on Latent
Continuum for Six Verbal IntelligenceSubtests

computed in order to study the nonuniqueness of this latent profile solution.

Only one restriction applied to all of the alternative solutions that were
tried, namely, that the resulting regressions be ascending for all subtests.
Within this restriction large changes in class sizes and in class averages could
be brought about, but never in such a way as to alter the ordering of curvatures
among the regressions. It should be added that only with very strained
assumptions about the spacing of the three classes along the latent continuum
could the regressions of both subtests 1 and 6 be made to curve in the same
direction.
Table 11 shows the fitted R~ implied by the approximate solution in
Table 13. A comparison of Tables 7 and 11 will reveal the similarities between
the fitted higher order manifest data for the two three-class latent profile
examples.
Discussion
There are two limitations to the foregoing latent profile examples that
should be discussed explicitly. These are in addition to the indeterminacy
250 PS.YCYIOMETRIKA

produced by the absence of higher order manifest data in the case of the two
empirical examples. The first is the lsck of a scale of measurement for the
latent continuum. For two classes this is unimportant, for it leaves only an
arbitrariness as to the origin and the distances of the two classes from that
origin. The problem of the relative distances between classes cannot arise
when there is only one such distance. With three-class solutions, however,
the problem of relative spacing is a critical one. Without some resolution of
it the regressions of tests on the latent continuum could not be drawn. Nor
could the shape of the distribution of positions along the latent continuum
be ascertained. In both of the present three-class solutions, the problem was
resolved by the arbitrary assumption of equal spacing of the classes along the
latent continuum. The regressions were drawn on that basis, and on the
same basis the latent distribution in each case became symmetric and approxi-
mately normal. Other assumptions about the underlying metric would have
led to different regressions and to different latent distributions. A separate
paper [9], stemming from some recent developments in latent structure
analysis [16], den.is further with this metric problem. It indicates one way in
which, with the aid of manifest product moments of still higher order, a
metric for the latent continuum can be made to emerge as an integral part
of the latent profile solution.
A second limitation of all latent profile examples in the present paper is
their unidhnensionality. It will be recalled that nothing in the development
of the latent profile equations restricts the number of underlying dimensions
within which the latent classes lie. Of course the two-class examples here can
be understood in terms of a single continuum, for that would be true of any
two-class case. The present three-class examples, however, are unidimensional
because of the special nature (homogeneous in content but graded in difficulty)
of the tests involved in them. Many three-class examples would require two
underlying dimensions for an adequate understanding of their psychological
meaning. In general, a q-class solution could require as many as (q - 1)
underlying dimensions for its interpretation. Subsequent work [cf. 10]
will deal with such multidimensional examples and with the problems of
dimensionality and metric that arise in their interpretation.
The reader may have noticed that in all four latent profile examples
no mention was made of a need for manifest data of order higher than the
third. Even for the two empirical examples a unique solution would not
have required the use of such higher order manifest data. These higher
order data therefore constitute a means for testing the assumption of higher
order within-class independence. This could be done by comparing the given
higher order data with the corresponding fitted values as generated from
the latent parameters by subsequent lines of equations (13) or (14). Alter-
natively it could be argued t h a t , if the solution never~ requires data above
third order, there is no need to postulate within-class independence beyond
 w . h. GIBSON 251

that order. From this parsimonious viewpoint, the latent profile equations
need extend no higher than third order, and it would be inappropriate to
think of using higher order data to test the fit of the model. However, it
might then be more important to test the adequacy of the solution by relating
it to variables not included in the original analysis. (An empirical example in
[8] gives an illustration of how this could be carried out in latent structure
analysis, where the very same argument over the use or non-use of higher
order equations not needed in a particular solution can be made.)
The latent profile equations that have been derived and illustrated here
are analogous to only one form of latent structure analysis--that knox~n
as the discrete class case. The analysis divides the sample into a small number
of discrete classes possessing second- and third-order within-class independ-
ence, and stops there. Other varieties of latent structure analysis, one of
which has already been referred to, have gone further in stipulating the
algebraic form of the regressions (the so-called trace lines [eft 15] of latent
structure analysis) or of the set of class sizes, or of both. Always, however,
the postulate of within-class independence is retained. Usually these further
restraints require within-class independence of higher than third order, so
that the corresponding higher orders of manifest data become directly
involved in the solution. Most of these variants of latent structure analysis
are readily translated into latent profile terms. In fact, the analogy between
the two models is so close t h a t almost whatever progress is made in the
various solutions for one model is convertible into a corresponding advance
for the other.

Conclusion
After outlining the derivation of the factor analysis and latent structure
models, this paper has shown how the latter can be generalized for analyzing
the interrelations among quantitathTe measures in a way that avoids some of
the troublesome problems of factor analysis. The resulting latent profile
n~ode] is applied to some simple fictitious and empirical data to illustrate its
use. Because such applications may seem to show some promise, it is perhaps
appropriate and not premature to conclude this paper merely by broadening
the reference for the admonition ([18], p. 70) with which it began.
It would be unfortunate if some initial success with the analytical
methods.., described here should lead us to commit ourselves to them with
such force of habit as to disregard the development of entirely different con-
structs that may be indicated by improvements in measurement and by
inconsistencies between theory and experiment.

REFERENCES

Ill Anderson, T. W. On estimation of parameters in latent structure analysis. Psycho-

metrika, 1954, 19, 1-10.
252 PSYCHOMETRIKA

[2] Carroll, J. B. The effect of difficulty and chance success on correlations between items
or between tests. Psychometrika, 1945, 10, 1-19.
[3] Davis, F. B. Fundamental factors of comprehension in reading. Psychometrika, 1944,
9, 185-197.
[4] Ferguson, G. A. The factorial interpretation of test difficulty. Psychometrika, 1941,
5, 323-329.
[5] Gibson, W. A. Applications of the mathematics of multiple-factor analysis to problems
of latent structure analysis. In P. F. Lazarsfeld et al., The use of mathematical models
in the measurement of attitudes. Res. Memo. No. 455, Rand Corp., 1951.
f6] Gibson, W. A. Applications of the mathematics of multiple-factor analysis to problems
of latent structure analysis. Unpublished doctoral dissertation, Univ. Chicago, 1951.
[7] Gibson, W. A. An extension of Anderson's solution for the latent structure equations.
Psychometrika, 1955, 20, 69~73.
[8] Gibson, W. A. Multiple factors and latent structure. Unpublished manuscript.
[9] Gibson, W. A. Nonlinear factor analysis, single factor case. Amer. Psychologist, 1955,
10, 438. (Abstract)
[10] Gibson, W. A. Nonlinear factors in two dimensions. Amer. Psychologist, 1956, 11,
415 (Abstract)
[11] Green, B. F. A general solution for the latent class model of latent structure analysis.
Psychometrika, 1951, 15, 151-166.
[12] Green, B. F. Latent structure analysis and its relation to factor analysis. J. Amer.
statist. Ass., 1952, 47, 71-76.
[13] Guilford, J. P. The difficulty of a test and its factor composition. Psychometrika, 1941,
6, 67-77.
[14] Lazarsfeld, P. F. The interpretation and computation of some latent structures. In
S. A. Stouffer et al., Measurement and prediction. Princeton: Princeton Univ. Press,
1950. Ch. 11.
[15] Lazarsfeld, P. F. The logical and mathematical foundation of latent structure analysis.
In S. A. Stouffer et al., Measurement and prediction. Princeton: Princeton Univ. Press,
1950. Ch. 10.
[16] Lazarsfeld, P. F. Some new results and problems in latent structure analysis. Un-
published manuscript.
[17] Lazarsfeld, P. F., et al. The use of mathematical models in the measurement of atti-
tudes. Res. Memo. No. 455, Rand Corp., 1951.
[18] Thurstone, L. L. Multiple-factor analysis. Chicago: Univ. Chicago Press, 1947.
[19] Thurstone, L. L. Note on a reanalysis of Davis' reading tests. Psychometrika, 1946,
11, 185-188.
[20] Wherry, R. J. and Gaylord, R. H. Factor pattern of test items and tests as a function
of the correlation coefficient: content, difficulty, and constant error factors. Psycho-
metrika, 1944, 9, 237-244.
Manuscript received 2/7/58
Revised manuscript received P/19/58