Sabol, F. R. (1998). What are we testing?: Content analysis of state visual arts achievement tests.

Visual Arts Research, 24(47), 1-12.

Critics suggest that test results may not indicate independently all types or levels of learning in
visual arts programs (Eisner, 1974, 1979; Hamblen, 1987; Higgins, 1989). It has been suggested
that, in addition to achievement tests, other assessment measures are needed to measure creative
problem solving abilities, higher order thinking skills, expressive production skills, divergent
thinking skills, and various types of learning that occur in visual arts education programs (Brandt,
1985; Day, 1985; Melville, Beard, Kearney, Roth, & Millman, 1987).
Prior to this recent attention on assessment, state level achievement testing in the visual
arts has been sporadic and idiosyncratic. Few examples of visual arts achievement tests have
been available for test developers to examine for content or format (Buros, 1978; Hausman, 1988;
Hopkins, Stanley, & Hopkins, 1990). Most recent tests have been created based on content from
state curriculum or proficiency guides and state-adopted textbook series (Hoepfner, 1988; Indiana
Department of Education, 1988; Sabol, 1990). These materials contain diverse objectives and
content. This diversity, combined with disagreement in the art education field concerning
appropriate content for visual arts curriculum and grade level requirements (Clark, Zimmerman,
& Zurmuehlen, 1987; Gardner & Grunbaum, 1986: Hoepfner, 1984; Maker, 1986; Wilson, 1988),
have led to equally great diversity in state level visual arts achievement tests across the country.
Purpose
Because wide disagreement exists about the content of art education and about what
content can be included on art achievement tests, analysis of state visual arts achievement tests
offers a means of describing what existing formats and content are being used in such tests. The
purpose of this study was to determine the form of test items and to identify content related to
subject themes, art history periods or styles, cultures, artists, works of art, and concepts and
vocabulary in test items. Findings of this study provide a research base that can assist test
developers and curriculum designers in their tasks.
Method
This study used quantitative research methods based upon a survey of all state
departments of education, interviews with state fine arts consultants, and content analysis of state
visual arts achievement tests. Content analysis methods recommended by Borg and Gall (1989),
Merriam (1988), Eisner (1981), and Krippendorf (1980) were used. Absolute frequencies and
descriptive statistics were used to summarize analysis of data.
Data Sources
A questionnaire devised by the researcher based on methods suggested by Borg and Gall
(1989) was sent to all state departments of education in the United States. Items on the
questionnaire were based on research questions that examined state visual arts curriculum content
and orientations as well as state visual arts test development, implementation, and use. States
with tests were asked to provide copies of tests, scoring guides, and other assessment materials.

F. Robert Sabol

Purdue University
What Are We Testing?: Content Analysis of State Visual Arts Achievement Tests

Author Identification Notes:

This study was done with support from the Getty Center for Education in the Arts
as part of the Getty Doctoral Fellowship Program. The author is grateful for this support
and acknowledges its role in completion of this study.

Abstract
Emphasis on accountability in visual arts education has encouraged assessment of
learning in art programs. Various forms of assessment are available and visual arts
achievement testing has been accepted as a reliable means of measuring student learning.
Interest in demonstrating accountability has led a number of state departments of
education to develop visual arts achievement tests. Although achievement testing has
been challenged, it has been used in many disciplines to produce evidence of levels of
student learning. Because there is wide disagreement about basic content for visual arts
curricula, state art achievement tests have been developed with a variety of objectives and
content from textbooks, curricula, and other sources. As a result these tests exhibit
significant diversity in form and content. This report describes the format and content of
such existing tests and clarifies what is being done nationally in assessment through use
of state visual arts achievement tests.

What Are We Testing?: Content Analysis of State Visual Arts Achievement Tests

Introduction
Standardized tests of achievement ... should be administered at
major transition points from one level of schooling to
another.... The tests should be administered as part of a
nationwide (but not Federal) system of State and local
standardized tests. (A Nation at Risk, 1983, p. 28)

Historically, the public has held schools accountable for providing students with
knowledge and developing skills and abilities. Significant public interest was generated
in public school accountability during the Sputnik era and more recently during the 1980s
due to international economic competition with the United States. Recent reports
including A Nation at Risk, (1983), Toward Civilization, (1988), America 2000: An
Education Strategy, (1991), and Goals 2000: Educate America Act, (1994) have
recommended that schools provide evidence of accountability through assessment.
In the past, assessment of learning in the visual arts has been through a variety of
means. Portfolios, critiques, reflective journals, essays, observations, interviews,
questionnaires, objective tests, achievement tests, and other “alternative” assessment
techniques have been recommended by Brandt (1987), Clark, Day, and Greer (1987),
Day (1985), Gardner and Brandt (1987), Hausman (1992), Hoepfner (1984), Wilson
(1971, 1996), and Zimmerman (1992).
 Educators from several state Departments of Education currently are developing
visual arts achievement tests to measure student learning in the visual arts (Council of
Chief State School Officers, 1985; Mamlin, 1986; Peeno, 1996; Peterson, 1991; Sabol,
1990). Writers for the American Psychological Association (1986) have defined an
achievement test as:
A test that measures the extent to which a person commands a
certain body of information or possesses a certain skill,
usually in a field where training or instruction has been
received. (p.89)
Driving development and use of these tests is the belief that test results help provide
evidence to the public of levels of learning being achieved by students in schools
(Stiggins, 1989).
Reviews with state fine arts consultants were conducted to verify and extend
questionnaire responses and to discuss test materials. All data requested and received are
available for public use and do not contain information that is considered to be personal,
private, or confidential. Validation of data was done through telephone interviews and an
inter-rater reliability study.
Findings
This study took place from 1992 through 1994. (Sabol, 1994). Questions were
examined that related to state visual arts curriculum content, compatibility of state visual
arts achievement tests and state curricula, and issues of concern to state fine arts
coordinators.
State Visual Arts Coordinators Questionnaire
Questionnaires were sent to 50 state departments of education fine arts
consultants and 34 were returned (68%). Of 16 states not submitting responses, eight did
not have visual arts consultants or the position was vacant during data collection.
Questionnaire responses were not supplied by five states, but these states are represented
in interview data. There was no questionnaire or interview data from three states. A total
of 12 states reported availability of state art achievement tests. Of this group two states
opted not to participate in the study. Of the ten participating states six reported that
creation of tests was mandated by state legislatures. Resources used in creation of tests
included state visual arts curriculum or proficiency guides (50%), local teacher input
(50%), state-adopted visual arts textbooks (30%), local curriculum guides (30%), artist
input (30%), and achievement tests from other disciplines (10%). Test item content was
selected from the National Assessment of Educational Progress models, state and local
curricula, artwork in local museums, committee consensus, and teacher suggestions. Test
development committees consisted of
art teachers from secondary school (28%), elementary school (23%), middle school
(19%), higher education (8%), administrators (6%), artists (5%), professional test
developers (5%), and museum educators (3%).

State Visual Arts Achievement Tests

A total of 26 state visual arts achievement tests were received from the ten states
participating in this study of visual arts achievement tests. Test item banks also were
submitted from two states for analysis. These provided a selection of items for use by
local school districts to construct their own achievement tests. Because these two states
did not have intact state achievement tests, content of these test item banks is not
included in this report. Tests from seven additional states were in preliminary stages of
development and consultants from these states felt it would be inappropriate to submit
their incomplete tests for analysis in this study. In the 26 tests received, a total of 100
assessment subsection components were identified. Objective tests were identified on all
26 tests. Objective tests, defined by Borg and Gall (1989) to be tests that are
uninfluenced or undistorted by the beliefs and biases of those who administer and score
them, were most frequently identified. The second most commonly identified tests were
those that required students to demonstrate studio or production skills. Studio or
production activities were found on 20 tests and attitude questionnaires and affective
questionnaires were identified 14 times each on five states’ tests. The highest number of
assessments for an elementary grade level was in the fifth grade with 13. There were 14
assessments at the seventh grade level and 18 at the eighth grade level. The highest
number of test assessments was at the eleventh grade with 19 tests.
Test Items
A total of 1,034 items were identified among all tests. Measures of central
tendency for all items on all tests indicated an average of 38.5 items with a median
number of 48 items and a mode of 48 items. Demographic items consisted of six percent
or 66 items. Of the remaining 968 items, 900 (93%) were multiple choice, 25 (5%) were
essay, 20 (2%) were true/false, and less than one percent each were completion,
production tasks, and short answer items that consisted of fewer than eight items each.
Criteria suggested by Dobbs (1992) were used for classification of items based on
their relationship to DBAE curriculum content. DBAE visual arts subject matter
knowledge was addressed in 713 items (69%). Affective, attitudinal, or student
demographic content was addressed in 321 items (31%). Of visual arts subject matter-
based items, 266 items (37%) examined knowledge of visual arts concepts and
vocabulary. There were 223 items (31%) related to art criticism skills or knowledge, 157
items (21%) about art history content, 41 items (6%) addressed art production
knowledge, and 26 items (3%) examined aesthetics knowledge.

Content Themes

Analysis of content-based items was done in order to determine the range and
degrees of similarity of content themes and concepts and vocabulary present on state
visual arts achievement tests. Content themes was defined as subjects or topics addressed
in visual arts instruction or represented in works of art. Content theme categories
included subject themes, art history periods or styles, cultures, artists, and works of art.
Visual arts concepts or vocabulary items included those defined as ideas, terms,
definitions, or abstractions of knowledge related to the study of visual arts.

Subject Themes

Subject themes were identified as the central ideas or thematic focus of test items
or works of art found on tests. In most cases, key subject theme words were identified.
Other cases required judgments about items with different terms, but synonymous
meanings. For example, items that used the term aristocracy and nobility were grouped
together. Of 1034 items 108 (10.4%) addressed subject themes. A total of 53 subject
themes were identified that ranged from 10 occurrences for one theme to a single
occurrence for 33 different themes.
_________________________
Insert Table 1 about here
_________________________

A total of 33 subject themes were found each in a single item. These represented
nearly one-third (31%) of all subject theme categories. The most commonly occurring
subject theme was moods. Of 108 subject theme items, ten (9.2%) addressed moods, but
did not specify particular moods. Test items in this category referred to the term moods
without reference to specific moods. Items that specified moods were grouped
separately.
Culture was the second most commonly occurring subject theme. Items in this
category addressed culture as a discrete subject theme. Culture was addressed in eight
items (7.4%). (References to specific cultures were not included in this category and
specific cultures were individually identified and categorized separately. See Table 3).
Items that specified functions of art were identified in seven items (6.4%). Items
that cited art as a historical record, form of communication, form of personal expression,
or other commonly accepted function of art were included in this category.
Both time and war occurred in five items each (4.6%) as subject themes.
References to the past, present, and future, and other time-related items were included in
this category. Items that referred to battle, combat, and armed conflict were included in
the war category.
Ceremonies and family relationships were found in four items (3.7%) each. Items
that addressed weddings, graduations, and other ceremonial events were included in the
ceremonies category. Family relationships items included references to mother and
daughter, father and son, brothers and sisters, and other family members.
A total of 33 subject themes were found in a single item each. These represented
nearly one-third (31%) of all subject theme categories.

Art History Periods or Styles.

State art achievement tests included items that referred to art history periods or styles.
Distinctions were not made between art history periods and styles, although several items
referred to periods as styles or styles as periods. For example, the term Greek was
referred to as both a period and a style in test items. Art history periods or styles were
identified in item content if they were named specifically. In some cases, judgment was
used to classify items that used differing terms, but were synonymous in meaning. For
example, items that used the terms realistic and representational were placed in the
realism category.
A total of 86 items (8%) addressed art history periods or styles. Test items
contained references to 26 art history periods or styles with a frequency range from nine
occurrences for one period or style to one occurrence each for seven art history periods or
styles.
__________________________
Insert Table 2 about here
__________________________

The most frequently occurring art history period or style was realism, identified in
nine items (10.4%). Eight items each (9%) referred to impressionism and surrealism .
Cubism and Greek were represented with eight items each (7%). Styles or periods
named abstract or futurism were found in five items each (6%). An additional 19
periods or styles were identified in four or fewer items. Western European or American
periods or styles predominated. References to other periods or styles were minimal or
non-existent.

Cultures

The concept of culture can be examined from a variety of perspectives. Comprehensive

views of culture might include political, gender, economic, geographic, social, or other
interpretations. For this study, cultures were identified from ethnographic and
anthropological stances as found in items with cultural references such as Acoma and
African.
References to cultures were found in 72 items (7%). A total of 27 culture
categories were identified with a frequency range of 11 items for one category to one
item each for 16 separate categories. Found in 11 items (15%), reference to the Japanese
culture was most frequently cited. The second most commonly identified culture was
Native American. It was found in eight items (11%). References to specific Native
American subgroups were not included in this category. Separate categories were created
to record the number of items for each Native American subgroup. When viewed as a
single group, combined items from the Native Americans and Native American
subgroups constituted the most frequently occurring cultural category with 30 items
(42%).
__________________________
Insert Table 3 about here
__________________________

Items that referred to African culture were identified seven times (10%) on tests.
Items in this category did not specify particular African cultures or subgroups.
References to specific African subgroups were not found on tests. Chinese and Eskimo
were the fourth most commonly occurring cultures addressed on tests. Each of these
categories contained six items, representing eight percent of all cultural items. The
remaining 22 cultures (46.8%) were found in four or fewer items each.

Artists

State art achievement test items frequently contained information about specific artists.
Artist’s names were recorded when found in test items or as part of informational labels
on works of art reproduced in test materials. Names of unidentified artists in test
materials were not included.
 Specific artists were named in 27% of all test items: 130 names of artists were
identified in 278 items. The frequency range of artists spanned from 15 items for one
artist to one item each for 65 artists.
__________________________
Insert Table 4 about here
__________________________

The artist most frequently named in test items was Picasso . He was named in 15
items (5.3%). References to Picasso were followed by Michelangelo and Van Gogh in
ten items each (3.5%). Found in eight items (2.8%) were references to Calder, followed
by seven items each (2.5%) for Hardigan and Moore . Items that referred to Da Vinci
and O’Keefe totaled six each (2.1%). Cassatt, Pollock, Rembrandt, and Rivera were
named in five items each (1.7%), while Durer, Hokusai, Homer, Munch, and Warhol
were named in four items each (1.7%). The remaining 111 artists represented 85% of the
130 artists named.
Of the 130 artists named 118 (91%) were of American or European nationalities.
Of all artists identified, 19 (15%) were women. Most artists found were painters, while
artists who create other art forms were seldom identified.

Works of Art

A total of 480 works of art were reproduced in state art achievement tests. These
works were classified into 15 categories based on the kind of art each represented.
Content of these categories ranged from 251 reproductions of paintings to one
reproduction of a postage stamp. Analysis revealed that of the 480 works of art there
were 251 (52%) paintings, 78 (16%) sculptures, 46 (10%) drawings, 24 (5%) prints, 18
(3.8%) masks, 16 (3.3%) fashion reproductions, 14 (3%) architecture examples, and 10
(2%) pottery examples. The remaining 23 works of art represented seven additional
categories.
Of 480 works of art reproduced on tests, written references were used to identify
283 (59%). Repeated references to specific works of art were found within grade levels
and across test components from individual states. There were 197 unidentified works of
art (41%) on tests. These generally were used to provide frames of reference for question
content, but titles of these works were not given. Written references on 91 items were
used to identify 71 titles of works of art.
__________________________
Insert Table 5 about here
_________________________

The most commonly occurring work of art identified by title was

Guernica, by Picasso. It was identified in eight test components or eleven percent of all
title references. Billboard, by Grace Hardigan, was the second most commonly identified
title; it occurred on four test components or in six percent of all title references.
Vincent Van Gogh’s Starry Night was identified in four percent of title
references and was present on three test components. Paintings by seven artists were
identified in two test components each or three percent of title references. These
included: Cow’s Skull - Red, White, and Blue by O’Keefe; Ghost Dance Shirt # 381 by
Havard; Mona Lisa by Da Vinci; Scorpion Pique Teapot by Shire; Self Portrait by
Hokusai; Self Portrait by Van Gogh; and Woman Reading by Cassatt. The remaining 61
titles were represented in one reference each on test components or 1.4 % of references to
titles.

Visual Arts Concepts and Vocabulary

Visual arts concepts and vocabulary were examined independently of art criticism,
aesthetics, art history, and production content areas. They were identified in the text of
items or in works of art reproduced on tests. In most cases, a key concept was identified
in the text of an item and assigned to a category. In other cases, judgment was used in
combining different terms whose meanings were synonymous. For example, items that
used the term formal and symmetrical to refer to balance were recorded in the same
category. Of 1,034 items analyzed, 451 (44%) contained references to visual arts
concepts or vocabulary. A total of 125 visual arts concepts or vocabulary terms were
identified. A frequency range of 16 references for one concept to one reference for 65
concepts resulted.
__________________________
Insert Table 6 about here
__________________________

The most frequently cited visual arts term was texture. It was found in 16 of 451
test items (4%). Color, shape, and style were found in 14 items each (3.1%) of concepts
items. A total of 13 items (3%) contained references to theme or main idea.. Balance,
line, and sculpture were identified in 11 items each (2.4%). Items that referred to rhythm
or movement totaled ten (2.2%). Visual arts terms or concepts found in nine items each
(2.0%) included formal/symmetrical balance, informal/asymmetrical balance, repetition,
and space . Of the remaining 112 concepts or vocabulary terms each was found in seven
or fewer items.

Discussion

Test Structure

Use of objective tests to measure student learning implies a set of assumptions

about the content and structures of tests (Hopkins, Stanley, & Hopkins, 1990). A
common assumption associated with achievement testing is that test content reflects a
common universe of knowledge presented in courses of instruction or in a field in
general. A second assumption is that achievement test items measure a representative
sampling of student knowledge, skills, and cognitive processes from the discipline. A
third assumption is that test items should be designed and structured to possess degrees of
difficulty that indicate levels of learning.
Tests that use a variety of item types such as multiple choice, essay, completion,
short answer, and so on, provide a more comprehensive indication of student knowledge
and skills than tests that use a limited selection of item types. For example, tests that
only use multiple choice items cannot examine significant types of learning that occurs in
visual arts programs. Tests in this study displayed a structural imbalance through
frequent and predominant use of multiple choice items. Multiple choice items do not
provide comprehensive assessment of all types of learning that may occur in visual arts
programs or require students to demonstrate higher level thinking skills. Use of a broader
range of item types would permit examination of a broader range of knowledge and
skills.
Achievement tests contained an average of 38.5 items per test. Given the number
of content areas of visual arts curricula, the amount of content in each area, and the
number of skills and processes in each area, increased numbers and types of items are
needed to measure adequately a representative sampling of student learning in art
criticism, aesthetics, art history, and production. Low numbers of items result in cursory
examination of a limited range of knowledge and skills. Significant learning is excluded
from the assessment process with limited numbers and types of items.

Curriculum Content Balance among Test Items

In suggesting DBAE content for curriculum Clark, Day, and Greer (1987)
identified aesthetics, art criticism, art history, and production as the basic disciplines of
the visual arts. They did not recommend that DBAE disciplines be given equal
instructional time in art programs, but they did suggest that equal attention be given to
each of the disciplines in curricula. Because each of the disciplines consist of
differentiated knowledge and skills, it is essential for achievement tests to address each
discipline with items that proportionately represent the content areas. Findings indicate
an imbalance in the number of items among these disciplines. The highest number of
content-based items was found in visual arts vocabulary and concepts knowledge with
266 or thirty-seven percent, followed by art criticism items with 223 or thirty-one
percent, and art history items with 157 or twenty-one percent. Only 41 items or six
percent of all content items addressed production knowledge and skills while the fewest
number of test items addressed aesthetics content with 26 items or four percent.
Knowledge, skills, and processes used in the creation of works of art have been
the focus of study in most art education programs. Acquisition, development, and
mastery of production knowledge, skills, and processes have been key elements in subject
matter-centered curricula in visual arts education. Nevertheless, low numbers of
production-related items on tests fail to reflect accurately the proportion of curricular
content, instructional time, and students’ use of production knowledge, skills, and
processes in visual arts education. Increased numbers of items are needed to measure
students’ levels of learning of production content.
Assessment of students’ production knowledge must not be confused with
production abilities. Production knowledge can be measured independently from
production skills. For example, knowing what the secondary colors are and that they are
made from combinations of the primary colors is different from being able to make the
secondary colors while using the primary colors of paint. An assumption of subject
matter-centered curricula is that basal knowledge can be identified for content areas.
Implied in this assumption is the belief that basic production knowledge exists. Further,
it is assumed that this knowledge can be taught and learned. It is erroneous to believe
that individuals who say they cannot produce works of art do not have knowledge of
skills and processes needed to produce them. Measurement of students’ levels of
production knowledge therefore should be a goal of assessment so that measurement of
production knowledge is not limited to demonstrations of such knowledge in studio
products.
Assessment of aesthetics learning is another issue of concern. Findings indicated
that of all content items only 26 (4%) addressed aesthetics content. Interview responses
from state fine arts consultants revealed that test developers found assessment of
aesthetics knowledge problematic. Use of multiple choice items did not provide students
with opportunities to compare differing philosophies related to aesthetics or allow
students to present their thinking about aesthetics questions. Evaluation criteria and
student subjectivity related to responses about aesthetics were or should be of equal
concern to test developers.
The presence of curricular content about aesthetics demands inclusion of items
related to assessment of knowledge about aesthetics and skills on tests. Failure to
provide assessment of learning about aesthetics diminishes the importance of the study of
aesthetics in visual arts programs and sends stakeholders a message that study of
aesthetics is unimportant. Problems associated with assessment of knowledge and skills
about aesthetics can and should be resolved. Alternative assessment methods such as
journals, critiques, and interviews provide possibilities for meaningful measurement of
learning in aesthetics.

Types of Art on Tests

Discrete classifications of types of art have existed throughout the history of art.
The variety of types of art has contributed to richness of artistic expression, meanings of
art, and roles art plays in our daily lives. Findings in this study suggested that more than
50% of all works of art reproduced on tests were classified as paintings which generally
are of European or American origin. Examples of paintings occurred three times more
frequently than any other type of art. Extensive use of examples of paintings ignores the
rich variety of other types of art from other cultures. Exclusion of additional types of art
suggests that other types are not as important or valid as is painting. Comprehensive
assessment of visual arts learning requires that students demonstrate knowledge of a
variety of art forms, media, functions, and purposes of art and be able to identify themes
and concepts present in these art forms. Examinations of student knowledge of a broad
variety of types of art are necessary in order to measure the depth of students’
understanding of the variety of types of art that exist.

Cultures on Tests

Findings indicated that works of art and artists named on tests primarily
represented western cultures. Only four works of art (7%) were created by artists from
non-western cultures. Contemporary emphasis on multiculturalism and global aspects of
art education curriculum requires that works of art and artists from a broad sampling of
cultures be studied and included on assessments. Representation of works of art from a
limited number of cultures is supported by a limited number of references to cultures in
test items. Curricula from eleven states that used tests for assessment purposes referred
to ten cultures (Sabol,1994), but related tests referred to only four. Test items referred to
27 cultures, although related curricula (Sabol,1994) did not contain references to 23 of
them.
Some culture labels that were used did not clearly identify recognized cultures.
African, Asian, European and similar labels in curricula offered nebulous cultural
categorizations from various geographic locations. Many distinct cultures exist within
these large geographic areas. Because a single cultures does not exist for these
classifications, these labels should be avoided and labels such as Asanti, Yoruba, and
Benin could be substituted for the African category.
In attempting to assess students’ knowledge of cultures, a small number of
cultures were commonly cited in test items. Cultures in this group are identified easily
through stereotypic images and artifacts that are commonly known. Japanese, Native
American, African and other so called cultures are found in this group. Cultures from
these groups may be emphasized because there are abundant resources and teaching
materials about them. Educational needs of students will not be met only through study
of a small number of popular cultures such as Japanese or various Native American and
African cultures. Study of a limited number of cultures will not permit students to fully
understand cultural diversity or identify similarities among cultures. It is advisable for
curriculum and test developers to include a broader array of cultures from within their
state, the country, and the world for study in visual arts programs. A broader selection of
cultures will expand to create greater understanding of multicultural education concerns.

Standardized and Authentic Assessment

Assessment may be thought of as the process of gathering information in order to

know what conditions exist and how they affect attainment of a desired goal or goals
(Stake, 1975). Assessment in visual arts education has been made problematic by the
varieties of learning that occurs in visual arts programs. Assessment of learning requires
use of measures that are compatible with types of learning being assessed (Hopkins,
Stanley, & Hopkins, 1990). Assessments must examine not only the knowledge and skill
levels of students, but also the variety of thinking skills and processes found in the visual
arts. Tests examined in this study relied on objective tests and primarily multiple choice
questions. This method of assessment provided limited evidence of types and degrees of
learning and prevented comprehensive measurement of student learning. Items on state
visual arts achievement tests examined generally did not require use of higher level
thinking skills or problem solving abilities. Such criticisms are not new and have
encouraged evaluators to use additional methods and measures to determine levels of
student achievement. Armstrong (1994), Hausman (1992), and Zimmerman (1992)
recommend use of authentic assessment measures, in addition to standardized measures,
to assess student art learning. Authentic assessments include an extensive array of
measures that can address a variety of knowledge, skills, and processes beyond those
measured by standardized tests. Findings of this study indicated that objective tests and
production activities made up nearly half of all components found on state visual arts
achievement tests. In isolation, these measures provide only partial indications of student
learning. When combined with authentic assessments they could provide broader
evidence of levels of learning. Expansion of assessment programs to include authentic
assessment methods could permit examination of a wider array of learning found in
visual arts programs and provide a more comprehensive profile of student achievement in
visual arts programs.

Conclusion

Examination of state visual arts achievement tests in this study revealed wide
ranges of format and content. The high level of diversity indicated apparent
disagreement about these assessment concerns in the field of visual arts education. Many
consider this diversity a sign of health for the discipline of art education and that it
reflects the diversity of the general field of art. Others believe high degrees of
conformity in assessments are restrictive and would place requirements on the field that
may not be reasonable in every educational setting or usable given a wide range of
students’ needs and abilities.
Within this mass of diversity there also exists a degree of agreement about format
and content needed for assessment of learning in visual arts education. Agreement about
formats is consistent with other disciplines such as those used in language arts, social
studies, and mathematics education and has been proven to provide valuable information
about student achievement. Areas of agreement about certain areas of test content suggest
that an emerging consensus about a basal core for art education may exist. Findings
about format and content in existing visual arts achievement tests examined in this study
indicate a need for continued study of this accepted type of assessment and that use and
study of alternative and authentic assessments may lead to more accurate profiles of
student achievement for the field of visual arts education.

References

American Psychological Association. (1985). Standards for educational and

psychological testing. Washington, DC: American Psychological Association.

Armstrong, C. L. (1994). Designing assessment in art. Reston, VA: National Art

Education Association.

Borg, W. R., & Gall, D. G. (1989). Educational research. New York: Longman.

Brandt, R. (1985). Overview. Educational Leadership, 43(2), 3.

Brandt, R. (1987). On assessment in the arts: A conversation with Howard

Gardner. Educational Leadership, 45 (4), 30-34.

Buros, O. K. (Ed.) (1978). The eighth mental measurements yearbook.

Highland Park, NJ: Gryphon Press.

Clark, G., Day, M., & Greer, W. D. (1987). Discipline-based art education:
 Becoming students of art. Journal of Aesthetic Education, 21(2), 129-193.

Clark, G., Zimmerman, E., & Zurmuehlen, M. (1987). Understanding art testing. Reston,
VA: National Art Education Association.

Council of Chief State School Officers. (1985). Arts education in the states: A survey of
state education policies. Washington, DC: Department of Education. U.S.
Government Printing Office.

Day, M. D. (1985). Evaluating student achievement in discipline-based art programs.

Studies in Art Education, 26(4), 232-240.

Eisner, E. (1974). Toward a more adequate conception of evaluation in the arts. Art
Education, 27(7), 2-5.

Eisner, E. (1979). The educational imagination: On the description and

evaluation of school programs. New York: Macmillan.

Eisner, E. (1981). Approaches to qualitative research. Review of Research in Art

Education, 13, 1-8.

Gardner, H., & Brandt, R. (1987). On assessment in the arts: A conversation with
Howard Gardner. Educational Leadership, 45(4), 30-34.

Gardner, H., & Grunbaum, J. (1986). The assessment of artistic thinking: Comments on
the national assessment of educational progress in the arts. Paper commissioned
by The Study Group on the National Assessment of Student Achievement.

Hamblen, K. A. (1987). What general education can tell us about evaluation in art.
Studies in Art Education, 28(4), 246-250.

Hausman, J. J. (1988). Back to the future: Reflections on present-day emphases in

curriculum and evaluation. Art Education, 41(2), 36-41.

Hausman, J. J. (1992). On the use of portfolios in evaluation: an editorial. Art Education,

45(1), 4-5.

Higgins, R. E. (1989). Approaches to evaluation in art education at the state level. Paper
commissioned by Minnesota State Department of Education.

Hoepfner, R. E. (1984). Measuring student achievement in art. Studies in Art Education,

25(4), 251-258.

Hoepfner, R. E. (1988). Assessing student achievement in art: A study of the feasibility

of achievement testing in art. Los Angeles, CA: Paper submitted to the Center for
 Education in the Arts. The J. Paul Getty Trust.

Hopkins, K. D., Stanley, J. C., & Hopkins, B. R. (1990). Educational and

psychological measurement and evaluation. Engelwood Cliffs, NJ:
Prentice-Hall.

Indiana Department of Education. (1988). Indiana fifth grade diagnostic visual art
achievement test. Indianapolis, IN: The Indiana Historical Bureau.

Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Beverly

Hills, CA: Sage.

Maker, J. C. (1986). Developing scope and sequence in curriculum. Gifted Child

Quarterly, 30(4), 151-158.

Mamlin, H.R. (1986). Testing, testing, 1, 2, 3, ... Design for Arts in Education, 88(1),
24-26.

Melville, S. D., Beard, J. G., Kearney, C. P., Roth, R., & Millman, J. (1987).
Current issues in testing, measurement, and evaluation. Washington, DC: Paper
submitted to Office of Educational Research and Improvement.

Merriam, S. B. (1988). Case study research in education: A qualitative

approach. San Francisco: Jossey-Bass.

National Commission on Excellence in Education. (1983). A nation at risk.

Washington, DC: U.S. Government Printing Office.

National Endowment for the Arts. (1988). Toward civilization: Overview from a report
on arts education. Washington, DC: National Endowment for the Arts.

Peeno, L. N. (1996). Status of the arts in the states. Reston, VA: National Art Education
Association.

Peterson, J. (1991). States assessment survey. Paper commissioned by the California

Department of Education.

Sabol, F. R. (1990). Toward development of a visual arts diagnostic

achievement test: Issues and concerns. In M. Zurmuehlen (Ed.),
Working papers in art education, 1989-90 (pp. 78-85). Iowa City, IA: The School
of Art & Art History of the University of Iowa.

Sabol, F. R. (1994). A critical examination of visual arts achievement tests from state
departments of education in the United States. Dissertation Abstracts
International, 9518525, Vol. 56, Issue 2A. (University Microfilms No. 5602A).
Stake, R. (1975). Evaluating the arts in education: A responsive approach.
Columbus, OH: Charles E. Merrill.

Stiggins, R. J. (1985). Improving assessment where it means the most: In the classroom.
Educational Leadership, 43(2), 69-74.

U.S. Department of Education. (1991). America 2000: an education strategy.

Washington, DC: U.S. Government Printing Office.

U.S. Department of Education. (1994). Goals 2000: Educate America act.

Washington, DC: U.S. Government Printing Office.

Wilson, B. (1971). Evaluation of learning in art education. In B. S. Bloom, J. T. Hastings,

& G. F. Madaus (Eds.), Handbook on formative and summative evaluation of
student learning (pp. 501-558). New York: McGraw-Hill.

Wilson, B. (1988). Art education, civilization and the 21st century: A

researcher’s reflections on the National Endowment for the Arts’ report to
Congress. Reston, VA: National Art Education Association.

Worthen, B. R., & Spandel, V. (1991). Putting the standardized test debate in perspective.
Educational Leadership, 48 (5), 65-69.

Zimmerman, E. (1992). Assessing students’ progress and achievement in art. Art

Education. 45(6), 14-24.
Table 1

Subject Themes Most Frequently Cited on State Art Achievement Tests: N = 108
__________________________________________________________
Subject theme f % of total items
__________________________________________________________
Moods 10 9.2
Culture 8 7.4
Functions of art 7 6.4
Time 5 4.6
War 5 4.6
Ceremonies 4 3.7
Family relationships 4 3.7
Animals 3 2.7
Children 3 2.7
Nature 3 2.7
Occupations 3 2.7
Power 3 2.7
Religion 3 2.7
Traditions 3 2.7
6 additional themes 2 each 1.9 each
33 additional themes 1 each 0.9 each
__________________________________________________________
Table 2

Art History Periods or Styles Most Frequently Cited on State Art Achievement Tests:
N = 86
__________________________________________________________
Art history period or style f % of total items
__________________________________________________________
Realism 9 10.4
Impressionism 8 9.3
Surrealism 8 9.3
Cubism 6 6.9
Greek 6 6.9
Abstract 5 5.8
Futurism 5 5.8
Expressionism 4 4.6
Post-Impressionism 4 4.6
Abstract Expressionism 3 3.4
Non-Objective 3 3.4
Pointillism 3 3.4
Pop 3 3.4
6 additional periods/styles 2 each 2.3 each
7 additional periods/styles 1 each 1.2 each
__________________________________________________________
Table 3

Cultures Most Frequently Cited on State Art Achievement Tests: N = 72

__________________________________________________________
Culture f % of total items
__________________________________________________________
Japanese 11 15.0
Native American 8 11.0
African 7 10.0
Chinese 6 8.0
Eskimo 6 8.0
Acoma 4 6.0
Egyptian 4 6.0
Iranian 3 4.0
Lakota 3 4.0
2 additional cultures 2 each 3.0 each
16 additional cultures 1 each 1.3 each
__________________________________________________________
Table 4

Artists Named on State Art Achievement Tests: N = 278

__________________________________________________________
Artist’s name f % of total items
__________________________________________________________
Picasso 15 5.3
Michelangelo 10 3.5
Van Gogh 10 3.5
Calder 8 2.8
Hartigan 7 2.5
Moore 7 2.5
Da Vinci 6 2.1
O’Keefe 6 2.1
Cassatt 5 1.8
Pollock 5 1.8
Rembrandt 5 1.8
Rivera 5 1.8
Durer 4 1.8
Hokusai 4 1.8
Homer 4 1.8
Munch 4 1.8
Warhol 4 1.8
11 additional artists 3 each 1.0 each
35 additional artists 2 each 0.7 each
68 additional artists 1 each 0.3 each
__________________________________________________________
Table 5

Works of Art Most Frequently Named on State Art Achievement Tests: N = 71

__________________________________________________________
Title of work of art, Number of test % of total
artist components references
__________________________________________________________
Guernica, Picasso 8 11.0
Billboard, Hardigan 4 6.0
Starry Night, Van Gogh 3 4.0
Cow’s Skull - Red, White
and Blue, O’Keefe 2 3.0
Ghost Dance Shirt #381,
Havard 2 3.0
Mona Lisa, Da Vinci 2 3.0
Scorpion Pique Teapot,
Shire 2 3.0
Self-Portrait, Hokusai 2 3.0
Self-Portrait, Van Gogh 2 3.0
Woman Reading, Cassatt 2 3.0
61 additional works of art 1 each 1.4
__________________________________________________________
Table 6

Visual Arts Concepts and Vocabulary Most Frequently Cited on State Art Achievement
Tests: N = 451
__________________________________________________________
Visual arts concept f % of total items
or vocabulary
__________________________________________________________
Texture 16 4.0
Color 14 3.1
Shape 14 3.1
Style 14 3.1
Theme/Main idea 13 3.0
Balance 11 2.4
Line 11 2.4
Sculpture 11 2.4
Rhythm/Movement 10 2.2
Formal/Symmetrical 9 2.0
Balance
Informal/Asymmetrical 9 2.0
Balance
Repetition 9 2.0
Space 9 2.0
7 additional concepts/terms 7 each 1.6 each
9 additional concepts/terms 6 each 1.3 each
10 additional concepts/terms 5 each 1.1 each
13 additional concepts/terms 4 each 0.8 each
15 additional concepts/terms 3 each 0.6 each
22 additional concepts/terms 2 each 0.4 each
40 additional concepts/terms 1 each 0.2 each
__________________________________________________________