Article

pubs.acs.org/accounts

Reflectance Hyperspectral Imaging for Investigation of Works of Art:

Old Master Paintings and Illuminated Manuscripts
Costanza Cucci,*,† John K. Delaney,‡,§ and Marcello Picollo†
†
Istituto di Fisica Applicata “Nello Carrara” - National Research Council (IFAC-CNR), Via Madonna del Piano, 10, 50019 Sesto
Fiorentino (Florence), Italy
‡
Scientific Research Department, National Gallery of Art, Fourth and Constitution Ave NW, Washington D.C. 20565, United States
§
Department of Bioengineering, School of Engineering and Applied Science, The George Washington University, Washington, D.C.
20052, United States

CONSPECTUS: Diffuse reflectance hyperspectral imaging, or reflectance

imaging spectroscopy, is a sophisticated technique that enables the capture of
hundreds of images in contiguous narrow spectral bands (bandwidth < 10 nm),
typically in the visible (Vis, 400−750 nm) and the near-infrared (NIR, 750−
2500 nm) regions. This sequence of images provides a data set that is called an
image-cube or file-cube. Two dimensions of the image-cube are the spatial
dimensions of the scene, and the third dimension is the wavelength. In this way,
each spatial pixel in the image has an associated reflectance spectrum. This “big
data” image-cube allows for the mining of artists’ materials and mapping their
distribution across the surface of a work of art.
Reflectance hyperspectral imaging, introduced in the 1980s by Goetz and co-
workers, led to a revolution in the field of remote sensing of the earth and near
planets (Goetz, F. H.; Vane, G.; Solomon, B. N.; Rock, N. Imaging
Spectrometry for Earth Remote Sensing. Science, 1985, 228, 1147−1152). In
the subsequent decades, thanks to rapid advances in solid-state sensor technology, reflectance hyperspectral imaging, once only
available to large government laboratories, was extended to new fields of application, such as monitoring agri-foods,
pharmaceutical products, the environment, and cultural heritage. In the 2000s, the potential of this noninvasive technology for
the study of artworks became evident and, consequently, the methodology is becoming more widely used in the art conservation
science field.
Typically hyperspectral reflectance image-cubes contain millions of spectra. Many of these spectra are similar, making the
reduction of the data set size an important first step. Thus, image-processing tools based on multivariate techniques, such as
principal component analysis (PCA), automated classification methods, or expert knowledge systems, that search for known
spectral features are often applied. These algorithms seek to reduce the large number of high-quality spectra to a common subset,
which allow identifying and mapping artists’ materials and alteration products. Hence, reflectance hyperspectral imaging is finding
its place as the starting point to find sites on polychrome surfaces for spot analytical techniques, such as X-ray fluorescence,
Raman spectroscopy, and Fourier transform infrared spectroscopy. Reflectance hyperspectral imaging can also provide image
products that are a mainstay in the art conservation field, such as color-accurate images, broadband near-infrared images, and
false-color products.
This Account reports on the research activity carried out by two research groups, one at the “Nello Carrara” Institute of Applied
Physics of the Italian National Research Council (IFAC−CNR) in Florence and the other at the National Gallery of Art (NGA)
in Washington, D.C. Both groups have conducted parallel research, with frequent interchanges, to develop multispectral and
hyperspectral imaging systems to study works of art. In the past decade, they have designed and experimented with some of the
earliest spectral imaging prototypes for museum applications. In this Account, a brief presentation of the hyperspectral sensor
systems is given with case studies showing how reflectance hyperspectral imaging is answering key questions in cultural heritage.

1. INTRODUCTION quality controls, agri-food, pharmaceuticals, and, as presented

Reflectance hyperspectral imaging was the result of intense here, to cultural heritage science.2−6
research activity started in the 1980s and led by Goetz within Hyperspectral imaging consists of a collection of images in a
several NASA Projects.1 The goal of this research was to sufficient number of contiguous spectral bands to allow
develop new technologies for remotely observing the earth, and generation of a reflectance spectrum at each pixel in the
the obtained result revolutionized the field of remote sensing.
Subsequently, hyperspectral imaging became well-established Received: January 29, 2016
and progressively spread out to other fields, such as industrial Published: September 28, 2016

© 2016 American Chemical Society 2070 DOI: 10.1021/acs.accounts.6b00048

Acc. Chem. Res. 2016, 49, 2070−2079
Accounts of Chemical Research Article

Figure 1. Scheme of the reflectance hyperspectral image-cube and spectra from two pixels, the floor and the blue surrounding Christ. Pacino di
Bonaguida’s Christ in Majesty with Twelve Apostles, Rosenwald Collection, NGA.

image. Typically, the acquired data set includes hundreds of instrumentation and the application of reflectance hyperspectral
images corresponding to very narrow bands (<10 nm) and imaging to the study of artworks are presented through
registered over an extended spectral range. The hyperspectral examples involving paintings and illuminated manuscripts.
data set is often referred to as an image-cube (Figure 1), since
each datum is identified by two spatial coordinates (the x and y 2. REFLECTANCE SPECTRAL FEATURES OF ARTISTS’
locating the pixel of the imaged surface) and one spectral MATERIALS AND IMAGE-CUBE PROCESSING
coordinate, the wavelength (λ). Thus, the image-cube includes ALGORITHMS
a unique set of spatial and spectral information on the imaged The Vis and NIR spectral regions are useful for analyzing
area. In earth surface observations, this sequence of images is polychrome surfaces by means of hyperspectral imaging. The
exploited to produce maps of distribution of minerals and reason for this is the presence of unique electronic and
vegetation, and the majority of instruments and exploitation vibrational features of artists’ materials in these regions. In the
tools have been developed for these applications. visible spectral region, electronic transitions typical of inorganic
Since the 1980s in the field of cultural heritage, spot pigments, such as charge transfer (e.g., ultramarine blue,
reflectance spectroscopy (1-D) in the visible (Vis, 400−750 Na8−10Al6Si6O24S2−4), conduction band (e.g., vermilion, HgS),
nm) and near-infrared (NIR, 750−2500 nm) regions has and ligand field (e.g., azurite, (CuCO3)2·Cu(OH)2) transitions,
proven to be effective for noninvasive pigment identification on dominate.28 Organic pigments also have electronic features that
paintings. Referred to as fiber optic reflectance spectroscopy can be used. For example, substituted anthraquinone pigments
(FORS), it has been successfully used to discriminate among (i.e., alizarin, C14H8O4, carminic acid, C22H20O13) have selective
and identify artists’ materials on polychrome surfaces.7−11 n−π* and π−π* absorption features from the delocalized
Beginning in 2000, new efforts were made to extend FORS into molecular orbitals.19,29 Visible reflectance spectra can also be
2-D spatial technique in order to map materials across the used for accurate colorimetric analysis.
painted surface, and hyperspectral cameras seemed the logical In the NIR region, some electronic transitions can be also
approach.12−19 Moreover, the growth of hyperspectral imaging present (i.e., iron oxides based on hematite, α-Fe2O3, and
in art examination boosted the adaptation of existing algorithms goethite, α-FeOOH), although this region is mainly dominated
for data processing and the development of new ones. Such by narrow spectral absorption features (∼10 nm) associated
multivariate analysis techniques, including principal component with the combination bands and overtones of the fundamental
analysis (PCA or PC), have since become more widespread in vibrational transitions that occur in the mid-IR range. For
the field of conservation science.19−26 example, vibrational transitions from hydroxyl and carbonate
This Account illustrates the salient achievements from the groups can be used to identify pigments (e.g., azurite and lead
groups at “Nello Carrara” Institute of Applied Physics of the white, (PbCO3)2·Pb(OH)2) or other paint materials (e.g.,
Italian National Research Council (IFAC−CNR) and the gypsum, CaSO4·2H2O).28 In addition, vibrational transitions in
National Gallery of Art (NGA) on adapting hyperspectral the 1600−2400 nm range can be used to discriminate among
imaging for the study of artworks. In the past decade they have organic materials, such as paint binders. Proteic media are
optimized hyperspectral imaging instrumentation and analysis characterized by absorption features associated with amide and
tools, originally developed for remote sensing, for applications carbonyl groups, whereas oils and fats are characterized by
in the cultural heritage field. This task required ad hoc solutions absorption features arising from methylene, methyl, and ethenyl
to address the constraints of examining inestimable art objects, functional groups.22,23 It has to be stressed, however, that all in
which are light- and thermal-sensitive. Furthermore, the fine situ methods share limitations and rely on other information for
details in artworks and the large number of artists’ materials accurate assignment.
require hyperspectral cameras capable of finer spectral and Analyses of the collected reflectance image-cubes can be
spatial sampling than those used in remote sensing. Other done in a variety of ways. In the simplest approach, the data set
issues arise such as the need to develop new image processing can be analyzed like spot reflectance spectra by extracting the
software, including the mosaicking of image-cubes, new pigment spectra individually, roving from site to site. However, this
identification and mapping methods, visualization, and time-consuming approach does not exploit the advantage of co-
archiving of the results.25−27 Here, key aspects of the collecting spatial and spectral information. This can lead to
2071 DOI: 10.1021/acs.accounts.6b00048
Acc. Chem. Res. 2016, 49, 2070−2079
Accounts of Chemical Research Article

Figure 2. IFAC−CNR scanner measurement campaign on the painting “Annunciazione e Cacciata di Adamo ed Eva dal Paradiso terrestre” by
Machiavelli Zanobi (1450 ca.) and scheme of the device.

missed information, for example: hyperspectral data collection of an earlier painting beneath the surface, or painted changes in
and analysis identified a region of zinc white (ZnO) in Picasso’s the final composition. Exploring pentimenti can help con-
Harlequin Musician that was missed during point analysis which servators understand how and why the changes were made.
only found lead white.19 Given the large number of reflectance They can lead to the discovery of prior paintings that have been
spectra collected (one for each pixel in the image), multivariate painted over. Also important is the identification of restored
statistics are needed to reduce the dimensionality of the data areas, or nonoriginal interventions containing inpaints that are
set. Thus, data reduction methods are of interest, including often removed in the conservation treatment process. The
techniques that enable the extraction of a basis set of spectra, or preparatory drawings beneath the painted surface (under-
“endmembers”, that best describe the majority of reflectance drawing) are also of interest.14,30,31 Visualizing underdrawing is
spectra in the image-cube.19,22−24 Most of the multivariant feasible using the NIR image-cube, since the infrared radiation
approaches start by using PCA to reduce the dimensionality of penetrates most pictorial paint layers.14,16,19,22,24,30,31
the data, resulting in a reduction from ∼200−800 spectral All of these features can be extracted from the image-cube and
bands down to 10−30 PC images. In a convex geometry enhanced through difference images or PCA. Such “enhanced-
approach, an algorithm known as the Pixel Purity Index (PPI) images” can be superimposed on the color image to elucidate
finds a subset of pixels in the hyperspectral cube whose spectra the relationship between them.
are the most unique and diverse. These pixels are then clustered Several types of hyperspectral imagers can be used to study
in the reduced multidimensional space defined by the number cultural heritage objects. The simplest design is a raster
of PC images that are retained. A multidimensional visualizer is scanning 1-D spectrometer, which suffers from long acquisition
often used to manually select pixel clusters that are well times; i.e., scanning 1 m2 at 1 mm sampling and 0.1 s dwell time
separated. The average spectra of each cluster define the takes ∼30 h.19 By using an imaging spectrometer with a 2-D
spectral endmembers. Maps of the endmembers can be made array detector instead, the acquisition time can be dramatically
using a variety of algorithms, such as the Spectral Angle reduced; i.e., scanning with imaging spectrometer having 1024
Mapping (SAM) algorithm, which measures the angle between pixels along the slit would take ∼3 min to image the 1 m2
the vectors that represent a reference spectrum and the painting at 1 mm spatial sampling. Thus, 2-D imaging
spectrum of a given pixel.22−24 Alternative approaches, like hyperspectral cameras have become the mainstay of the field.
expert knowledge systems, use preidentified spectral features, To cover the Vis-NIR range, two separate camera systems have
such as slopes of electronic transitions or spectral vibrational to be employed. Si-based CCD or CMOS detectors are used
features, to classify the spectra.27 Assignment of pigments is for the 350−1000 nm range, whereas InGaAs (900−1700 nm)
best done by identifying spectral features whose chemical origin and InSb or MCT arrays are used for the1000−2500 nm range.
is known rather than direct “matching” typical in Raman or Another difference between the hyperspectral imagers used
FTIR spectroscopy. in remote sensing and cultural heritage are the light sources and
The image-cube can also be exploited directly by shifting light levels used. Since artists’ materials are photosensitive,
through the image stack to find images that best reveal specific although to differing degrees, care is needed to minimize
traits and features not completely visible to the naked eye. Of exposure to excessive light levels or long exposure times.32,33
particular interest is the visualization of pentimenti, visible traces Remote sensing hyperspectral cameras are typically optimized
2072 DOI: 10.1021/acs.accounts.6b00048
Acc. Chem. Res. 2016, 49, 2070−2079
Accounts of Chemical Research Article

Figure 3. (a) Color image reconstructed from IFAC−CNR hyperspectral data acquired on the painting Tondo Doni, dated 1505−1506, by
Michelangelo Buonarroti, Uffizi Gallery (Florence) collection (Inv. 1890 n.1456). (b) Color detail. (c) Same detail at 880 nm. (d) Detail of a
pentimenti highlighted from the PC2 image obtained from the 700−800 nm range. Image permissions granted by the Italian Ministero dei beni e
delle attività culturali e del turismo (MIBACT): All reproduction rights are reserved.

for solar illumination and higher light intensity levels compared Determination of the color accuracy of the IFAC−CNR
to museum lighting. Thus, hyperspectral cameras constructed scanner using calibrated reflectance colorimetric standards
for studying artworks need to have increased sensitivity to yielded an average color difference of ΔE = 1.4 ± 0.6
operate at lower illumination levels, as well as unique (calculated with ΔEab76 formula), which is below the visually
illumination systems that filter UV and thermal radiation. appreciable threshold of color difference.39 The combination of
high color accuracy and high spatial sampling (0.1 mm) that
3. THE IFAC−CNR REFLECTANCE HYPERSPECTRAL can be obtained from the IFAC−CNR image-cubes makes
SCANNER AND ITS APPLICATIONS possible magnification of details without any loss of quality.
In 2003, the IFAC−CNR research group designed and Thus, for example the pattern of paint cracks (craquelure),
assembled a transportable hyperspectral scanner operating in which is one of the finest details of interest for conservation
the 400−900 nm range, optimized for acquiring data on large documentation, can be visualized.41 Monitoring the craquelure
areas of paintings (100 × 100 cm2) with high spatial sampling and its changes over time can provide information about the
(100 μm).15,16,34 Subsequently, the system was upgraded to deterioration of the painting surface and also insight into the
extend the spectral sensitivity to 1700 nm and applied to other artists’ technique and materials.
case studies.35,36 The imaging heads and illumination systems A significant case-study investigated with the IFAC−CNR
of this hyperspectral push-broom scanner (Figure 2) are scanner was Michelangelo Buonarroti’s Tondo Doni (“The Holy
“pushed across” the painting to build up the image-cube one Family”) at the Uffizi Gallery, Florence. In 2008, the OPD
spatial line at a time. (Opif icio delle Pietre Dure) led an extensive noninvasive
The system uses two commercial prism-grating-prism (PGP) measurements campaign to study this masterpiece. IFAC−
line-spectrographs connected respectively to a high-sensitivity CNR scanner measurements, performed in the 400−900 nm
Si CCD (a 1344 × 1024 detector array with 6.45 μm pixels range, yielded high-resolution color accurate RGB images, PC
operating in 2× binning mode, effectively giving an array size of images, as well as reflectance spectra. When combined with the
672 × 512 with 13 μm pixels) and InGaAs (640 × 512 pixels, data from other analytical techniques, these results provided the
with 20 μm pixel size) camera. An f/8 telecentric objective is identification of artists’ materials and documented the presence
used to minimize image blurring and geometric distortion from of hidden details.42 Figure 3a shows the overall calibrated RGB
nonplanar surfaces. The illumination module uses two long color image, and a magnified detail is reported in Figure 3b.
cylindrical lenses fed by inline fiber-optic illuminators that The same detail obtained from the 880 nm image (Figure 3c)
collimate the light and symmetrically illuminate a narrow region shows a refinement of the original drawing along with
on the painting from 45° relative to the painting normal. This pentimenti in the right-hand of the Virgin holding the Child,
minimizes the portion of the painting illuminated to the strip seen as a lighter tone on the upper part of the hand. This
being imaged by the sensor head as it moves along. The 2 × refinement is made clearer in the second PC image obtained
45°/0° configuration illumination/observation geometry is from the 700−800 nm portion of the image-cube (Figure 3d).42
essential to provide calibrated RGB images and colorimetric Exploitation of NIR hyperspectral images allows inspection
values (i.e., CIEXYZ, CIEL*a*b*, sRGB, etc.) as recommended of the subsurface paint layers and underdrawing in paintings. In
by CIE (Commission Internationale de l’Eclairage).37−40 The art conservation, NIR reflectance imaging (known as infrared
scanner acquires 750 images in the 400−1700 nm range with reflectography, IRR) is commonly used to visualize details
spectral resolution of ∼2 nm in the Vis and ∼8 nm in the NIR. beneath the painted surface such as underdrawing. However,
The combination of high scan rate and small area illuminated at traditional IRR is a broadband imaging technique whose
a time ensures the light exposure levels are compatible with the
spectral range is generally defined by the sensitivity range of the
recommended limits in museums.32,33
detector array used (900−1700 nm for InGaAs). The
3.1. High-Quality Imaging Documentation acquisition of such broadband images often results in decreased
Accurate color capture is required for any imaging sensor that contrast of the underdrawing by the partially penetrated paint
provides archival quality documentation of works of art. layers, thus degrading the visualization of underdrawing or
2073 DOI: 10.1021/acs.accounts.6b00048
Acc. Chem. Res. 2016, 49, 2070−2079
Accounts of Chemical Research Article

other details (i.e., compositional paint changes). Hyperspectral

data acquired in the NIR range allow selecting the best spectral
bands to visualize with a better contrast these drawings and
hidden details, or to create image products that enhance their
visualization.14
An example of improved IRR images can be seen in the NIR
data collected from a panel painting entitled “Annunciazione e
Cacciata di Adamo ed Eva dal Paradiso terrestre” (ca. 1450) on
display in the San Martino a Mensola church, in Florence. This
painting is attributed to the Florentine painter Machiavelli
Zanobi, a follower of Beato Angelico. The conservative
intervention was directed by Dr. Lia Brunori in 2012 for an
exhibition at the Uffizi Gallery, and utilized noninvasive analysis
techniques to gain new insights. The large painting (195 × 186
cm2) was completely imaged with the IFAC−CNR scanner in
the 900−1700 nm range, whereas the figure of the Virgin was
imaged in the 400−1700 nm. A key goal of the measurements
campaign was to acquire the best quality image of the
preparatory underdrawing, never before seen. The image at
1335 nm from the mosaicked NIR image cubes was found to
give the best visualization of the preparatory drawing (Figure
4a). This NIR image documents the fine and subtle
underdrawing that aids in understanding the artist’s style and
technique.
The NIR image cubes not only provide high quality IRR, but
also offer the possibility to enhance features not readily
apparent in a traditional IRR. For example, when the color
detail (calculated from the Vis image-cube, Figure 4b) is
compared to the “difference image” (Figure 4c), a dark area on
the Virgin’s forehead clearly emerges, indicative of a prior
restoration. Likely this restoration encompasses the pictorial
layers down to the preparation ground. Indeed, reflectance
spectra from a site on the dark patch (P1) and outside the dark
patch (P2) (Figure 4d) show that in P2, the spectral features of
gypsum from the 2ν(OH−) features at 1445, 1490, and 1535
nm can be seen, and thus confirm gypsum was used as the
ground layer for the painting. Conversely, these absorption
features are not present in the spectra from P1, taken from the
dark site. Here the absorption features are shifted with the main
band appearing at 1425 nm, shifted approximately 30 nm from
that of gypsum, confirming another fill material was used. This
is one of several examples showing this type of information Figure 4. Images from the NIR image-cube acquired on the
cannot be obtained using broadband IRR.14,22 Machiavelli Zanobi painting. (a) Overall image at 1335 nm. (b)
Detail color image from Vis image-cube. (c) Difference image
3.2. Extracting Reflectance Spectra from Small Details produced by subtracting the 1470 nm image from the 1445 nm
Due to the high spatial sampling of the IFAC−CNR image. (d) Reflectance spectra extracted from the image-cube at the
points P1 and P2 (see c).
hyperspectral sensor, reflectance spectra from areas as small
as 100 μm can be collected on fine polychrome miniatures or
similar artworks. The use of hyperspectral imaging to provide challenging by reflectance spectroscopy, thus supplementation
information on the palette of six 18th century small paintings with other nondestructive techniques is often useful. However,
by Gaspare Lopez, known as “Gasparo of the Flowers”, is spot techniques such as portable XRF (unless a polycapillary
presented.35 These small (∼10 × 12 cm2) oil paintings were optic is used) or FTIR cannot be used since they commonly
made on copper plates. Despite their reduced dimensions, they have spot sizes larger than 1 mm. Only micro-Raman
have striking chromatic richness. The color images and the VIS spectroscopy could have provided a certain identification of
reflectance spectra extracted from the image-cube of two this yellow, but fluorescence from the oil medium might have
adjacent points (P1 and P2, about 2 mm distant) painted with masked the Raman features of the pigment.
yellow and red pigments, respectively, are reported in Figure 5.
The red was identified as vermilion given the spectrum has 4. NGA REFLECTANCE HYPERSPECTRAL IMAGING
the typical S-shape characteristic of band-to-band transitions in SYSTEMS AND EXAMPLE APPLICATIONS
semiconductors, with the inflection point around 600 nm as The NGA reflectance hyperspectral systems were constructed
determined from the first derivative of the spectrum.19 The with several unique features to increase their utility for
yellow was tentatively identified as Naples yellow, Pb3(SbO4)2, analyzing artworks. These designs were the outgrowth of
by its spectral shape. Identification of yellow pigments is experiments begun in 2007 with early generation scanning
2074 DOI: 10.1021/acs.accounts.6b00048
Acc. Chem. Res. 2016, 49, 2070−2079
Accounts of Chemical Research Article

Figure 5. (a) RGB color image reconstructed from the IFAC−CNR hyperspectral data acquired on an oil-on-copper painting (11.8 × 8.8 cm2) by
Gaspare Lopez, Uffizi Gallery (Florence) collection (Inv. 1890 n. 4603). (b) Detail indicating points P1 and P2 corresponding to the yellow and red
pigments identified. (c) Reflectance spectra obtained as average of a 3 × 3 px area (0.25 × 0.25 mm2) used for identifying the pigments. Image
permissions granted by the Italian Ministero dei beni e delle attività culturali e del turismo (MIBACT): All reproduction rights are reserved.

Figure 6. Generic scheme (left) and images of the NGA Hyperspectral cameras, Vis (bottom-right) being used to collect an image-cube of a detail of
Giorgio Vasari, Page from Libro de’ Disegni, c. 1480−1504, Woodner Collection, Patrons’ Permanent Fund, NGA, and exNIR (top-right), being used
to image Rembrandt’s Self-Portrait, (top-right) c. 1659, Andrew W. Mellon Collection, NGA.

hyperspectral cameras operating from 400 to 1700 nm.19 The scripts, which are more light sensitive.33 Finally, the third
first key feature is portability, the ability to collect image cubes feature involved extending the spectral range to 2500 nm with
in the visible through NIR (400−1700 nm) without having to 2.8 nm spectral sampling, which allows for the mapping of paint
translate the cameras. This was achieved by integrating a scan binders and improved pigment identification.
mirror into the hyperspectral camera (Figure 6) to acquire tens Like the IFAC−CNR cameras, the NGA instruments utilize
to hundreds of image-cubes that are mosaicked together with prism-transmission grating spectrometers. The visible hyper-
custom software. The second feature was the use of low noise, spectral camera is coupled to a backside illuminated Si EMCCD
low dark current, and high sensitivity 2-D array detectors. This [1024 (spatial) by 1024 (spectral) pixels, 13 μm pixel size,
allows for operating at light levels of 1000−2000 lx, making 400−950 nm, 2.5 nm sampling] having a quantum efficiency of
possible the examination of paintings and illuminated manu- >90% (Figure 6). The extended NIR camera (1000−2450 nm)
2075 DOI: 10.1021/acs.accounts.6b00048
Acc. Chem. Res. 2016, 49, 2070−2079
Accounts of Chemical Research Article

Figure 7. (a) Color image of the Virgin Mary panel by Cosimo Tura (31 × 12.4 cm2, 1475), Samuel H. Kress Collection, NGA. (b) False-color map
created with the SAM algorithm, which identifies pixels in the hyperspectral cube whose reflectance spectra match the endmember spectra shown in
(c). Pixels are assigned to an endmember if the angle is less than or equal to the tolerance angle.22

uses a digital ROIC InSb detector array [1024 (spatial) by 1280 reflectance spectra collected, multivariate statistics were used
(spectral) pixels, 12 μm pixel size, 2.8 nm sampling] operating to reduce the dimensionality of the data set. In this example,
at 70 K. A telecentric relay optic with an external pupil is used PCA reduced the dimensionality from 536 to 13. The PPI
to match the cold stop of the F/D 2.3 InSb camera. The cold algorithm was used to find ∼10 000 spectrally pure pixels for
stop of the IR camera dewar and the use of two band-pass cold clustering in the 13-dimensional PC space. The reflectance
filters (1000−2450 nm light) limits midwave infrared radiation spectra identified by the PPI algorithm were manually clustered
from reaching the detector and thus improving signal-to-noise in PC space, resulting in eight spectral endmembers that best
at long exposures (100−200 ms per line). These compact represented the painting. Mapping the location of endmembers
portable systems can be operated in two modes, either by push- was performed using the SAM algorithm. This algorithm
broom scanning (by moving the painting in front of the returns a metric for the degree of match between the spectrum
hyperspectral cameras by using a computer-controlled 2-D at each pixel and the reference endmember spectra. The
easel capable of scanning an area of 1.5 m2) or whiskbroom resulting hyperspectral maps show the spatial distribution of
scanning (using an internal scan mirror).19 The data acquired these endmembers. The eight-reflectance spectral endmembers
by both cameras are calibrated to apparent reflectance and (Figure 7c) were found to well describe the majority of the
reconstructed in a continuous image cube from 400 to 2500 nm painting as seen in the map given in Figure 7b. Identification of
with 750 images, 2.5−2.8 nm spectral sampling, and 160 μm the pigments present in the endmember spectra was done by
spatial sampling. Using these cameras, together with an InGaAs comparing their spectral features with those of reference spectra
version (967−1680 nm, 3.4 nm sampling), paintings as large as from artist pigments. The endmember spectra that could be
1.7 × 1.8 m have been imaged and analyzed. uniquely assigned include the following: gypsum for the ground
NGA hyperspectral camera systems and mapping algorithms layer, azurite for the blue sky, natural ultramarine for Mary’s
have been used to determine the spatial distribution of artists’ robe, iron oxides for the hair and face, and an insect-based
materials or “maps” based on their spectral features. Maps of organic red lake for Mary’s tunic. The use of lead white and the
artists’ materials from works by early Italian artists to modern pigment used for the landscape could not be conclusively
painters have been completed.11,19,22−24,27 identified with hyperspectral reflectance imaging in this case.
The results obtained from examining a 15th century panel The identification of lead white relies on the presence of a
painting by Cosimo Tura are reviewed here. The painting hydroxyl feature at 1447 nm, which is from hydrocerussite,
depicts the Virgin Mary (Figure 7a), and is one of four panels 2Pb(CO3)·(OH)2, the basic form of cerussite (PbCO3), a lead
that comprise The Annunciation with Saint Francis and St. Louis carbonate. Before the 17th century, lead white could be a
of Toulouse, ca.1475.22 Image cubes (400−1680 nm) of the mixture of cerussite and hydrocerussite, making the 1447 nm
panel were collected using two hyperspectral cameras and feature less noticeable. Gypsum has three hydroxyl features
registered together.26 Because of the large number of noted above and the first is at 1447 nm. Assignment of lead
2076 DOI: 10.1021/acs.accounts.6b00048
Acc. Chem. Res. 2016, 49, 2070−2079
Accounts of Chemical Research Article

white in the presence of gypsum is possible if the depth of the

first band at 1447 nm shows a larger depth than what is
expected for gypsum alone. The pigment used for the landscape
was tentatively identified as umber (iron and manganese oxides,
a natural brown earth pigment) by its spectral shape. However,
the collection of XRF element maps showed copper is present
in the landscape, an element not found in umber. The best
match to both the XRF and reflectance results was a degraded
green copper resinate, Cu(C19H29COO)2, which is known to
turn brown over time.22 Thus, what appears as a rocky
landscape was once green. The use of complementary methods,
such as hyperspectral reflectance imaging and XRF point
analysis or scanning measurements, has been important for
definitive pigment assignments, especially for those pigments
that have simple reflectance spectra, such as a single transition
edge seen in some yellow pigments.
By extending further into the NIR, specifically from 1700 to
2450 nm, the identification and mapping of some paint binders
has been possible. Two papers have shown the weak absorption
features associated with oils, proteins, and carbohydrates can be
used to identify drying oils, egg yolk tempera, wax, protein-
based glues, and gums.23,24 For example, the specific use of egg
yolk and animal skin glue was mapped using NIR hyperspectral
imaging in a Cosimo Tura painting consisting of four panels, Figure 8. NIR image cube analysis of Lorenzo Monaco’s Praying
one of which is the panel depicting the Virgin Mary as Prophet, Rosenwald Collection (1410−1413). (a) Color image. (b)
discussed above. The maps showed the artist selectively used Projection map of 10-D clustering: the green, blue, and red points
animal skin glue for blue pigments and egg yolk for an insect- define the clusters used for the three endmember spectra in (c). The
based red lake pigment.24 These findings match prior results false-color image (d) shows the pixels whose spectra do not deviate
obtained by HPLC analysis of microsamples taken from the more than the tolerance angle.23 The red areas show evidence for the
panels.24 presence of egg yolk, and the green and blue do not. The green areas
show vibrational features assignable to azurite that was mixed with an
A more challenging and unexpected case was the mapping of unknown yellow to create the green initial.
paint binders in a 15th century illuminated parchment foil by
Lorenzo Monaco, where microsampling was not possible.
While gum Arabic or protein would have been the expected
paint binders’, evidence for the use of egg yolk tempera, used features useful for the identification of artist’ materials.
for panel painting during this time period, was found by its However, preliminary studies show the concerns of heating,
absorption feature at 2309 nm. This vibrational feature is challenges in spectral interpretation, and high camera costs, and
associated with the methylene functional groups found in egg suggest that progress will be slower than in the visible and near-
yolk lipids.23,24 The egg yolk tempera binder was only found in infrared.46 Nevertheless, reflectance hyperspectral imaging is
the painted figure of the prophet (Figure 8).23 finding its place as the starting point for diagnostic study,
specifically because it can access the entire surface of a painting.

■
5. CONCLUSIONS
Since the 2000s, standoff reflectance hyperspectral imaging has AUTHOR INFORMATION
emerged as a promising technique for the examination of Corresponding Author
paintings and works on paper. In the last two decades the
complementary research carried out at IFAC−CNR and NGA *E-mail: [email protected].
laboratories, as well as others, has shown the utility of Notes
hyperspectral imaging to not only inspect and document The authors declare no competing financial interest.
artworks, but to provide improved visualization of under-
drawings and compositional paint changes, as well as to identify Biographies
and map artists’ materials in situ. Costanza Cucci got Physics “Laurea” and Ph.D. in Conservation
Works of art are inherently challenging to study because of Science. She is a researcher at IFAC−CNR. Her research focus is on
the diversity of materials used, the stratigraphy of the paint spectroscopic techniques applied to materials characterization in the
layers present, and the range of object sizes encountered. While fields of cultural heritage, environmental monitoring, and safety food
progress has been made in developing hyperspectral cameras to controls.
capture image-cubes of art objects, including large frescoes and
mural paintings at archeological sites, challenges remain.43,44 John K. Delaney, Ph.D., is the Senior Imaging Scientist at the NGA,
Among these are the need for better algorithms specific for where his research focuses on the application of remote sensing
automatic mapping and separation of pigment mixtures.27,45 To imaging methods for the study of works of art.
support this, reflectance spectral databases tailored to cultural Marcello Picollo, Ph.D., is a researcher at IFAC−CNR. His interests
heritage are required. The development of detector arrays and include color measurement, Vis-NIR hyperspectral imaging, and spot
dispersive optical systems offer extending the spectral range in size UV−Vis-IR spectroscopic investigations of 2D polychrome
the UV and mid-IR, regions known to be rich in spectral objects.

2077 DOI: 10.1021/acs.accounts.6b00048

Acc. Chem. Res. 2016, 49, 2070−2079
Accounts of Chemical Research Article

■ ACKNOWLEDGMENTS
J.K.D. thanks K. Dooley for help and acknowledges support
for Arts and Archaeology. Proc. SPIE 5857; Pezzati, L., Salimbeni, R.,
Eds.; SPIE: Munich, 2005; pp 58570M−58570M-8.
(16) Picollo, M.; Bacci, M.; Casini, A.; Lotti, F.; Poggesi, M.; Stefani,
from the Andrew W. Mellon and Samuel H. Kress Foundations L. Hyperspectral image spectroscopy: a 2D approach to the
and the National Science Foundation (1041827). C.C. and investigation of polychrome surfaces. In Proc. Conservation Science,
M.P. are extremely grateful to their colleagues A. Casini and L. Milan, May 10−11, 2007; Townsend, J., Toniolo, L., Capitelli, F., Eds.;
Stefani for the development of the IFAC−CNR scanner and to Archetype Publications: London, 2008; pp 162−168.
the colleagues of the former Soprintendenza SPSAE e per il (17) Casini, A.; Lotti, F.; Picollo, M.; Stefani, L.; Buzzegoli, E. Image
Polo Museale della città di Firenze and the Soprintendenza Spectroscopy mapping technique for non-invasive analysis of
BAPSAE di Firenze Pistoia e Prato of the Italian MIBACT for paintings. Stud. Conserv. 1999, 44, 39−48.
having allowed the access and measurements on the artworks. (18) Bacci, M.; Casini, A.; Cucci, C.; Muzzi, A.; Porcinai, S. A study

■
on a set of drawings by Parmigianino: integration of art-historical
analysis with imaging spectroscopy. J. Cult. Herit. 2005, 6, 329−336.
REFERENCES (19) Delaney, J. K.; Zeibel, J. G.; Thoury, M.; Littleton, R.; Palmer,
(1) Goetz, F. H.; Vane, G.; Solomon, B. N.; Rock, N. Imaging M.; Morales, K. M.; Rie, E. R. D. L.; Hoenigswald, A. Visible and
Spectrometry for Earth Remote Sensing. Science 1985, 228, 1147− Infrared Imaging Spectroscopy of Picasso’s Harlequin Musician:
1152. Mapping and Identification of Artist Materials in situ. Appl. Spectrosc.
(2) Gowen, A. A.; O’Donnell, C. P.; Cullen, P. J.; Downey, G.; Frias, 2010, 64, 584−594.
J. M. Hyperspectral imaging − an emerging process analytical tool for (20) Baronti, S.; Casini, A.; Lotti, F.; Porcinai, S. Multispectral
food quality and safety control. Trends Food Sci. Technol. 2007, 18, imaging system for the mapping of pigments in works of art by the use
590−598. of principal-component analysis. Appl. Opt. 1998, 37, 1299−1309.
(3) Tatzer, P.; Wolf, M.; Panner, T. Industrial application for inline (21) Mansfield, J. R.; Sowa, M. G.; Majzels, C.; Collins, C.; Cloutis,
material sorting using hyperspectral imaging in the NIR range. Real- E.; Mantsch, H. H. Near infrared spectroscopic reflectance imaging:
Time Imaging. 2005, 11, 99−107. supervised vs. unsupervised analysis using an art conservation
(4) Hamilton, S. J.; Lodder, R. A. Hyperspectral imaging technology application. Vib. Spectrosc. 1999, 19, 33−45.
for pharmaceutical analysis. In Biomedical Nanotechnology Architectures (22) Dooley, K. A.; Conover, D. M.; Glinsman, L. D.; Delaney, J. K.
and Applications. Proc. SPIE 4626; Bornhop, D. J., Dunn, D. A., Complementary Standoff Chemical Imaging to Map and Identify
Mariella, R. P., Murphy, C. J., Nicolau, D. V., Nie, S., Palmer, M., Artist Materials in an Early Italian Renaissance Panel Painting. Angew.
Raghavachari, R., Eds.; SPI: San Jose, CA, 2002; pp 136−147. Chem. 2014, 126, 13995−13999.
(5) Kellicut, D. C.; Weiswasser, J. M.; Arora, S.; Freeman, J. E.; Lew, (23) Ricciardi, P.; Delaney, J. K.; Facini, M.; Zeibel, J. G.; Picollo, M.;
R. A.; Shuman, C.; Mansfield, J. R.; Sidawy, A. N. Emerging Lomax, S. Q.; Loew, M. H. Near Infrared Reflectance Imaging
technology: hyperspectral imaging. Perspect. Vasc. Surg. Endovasc. Spectroscopy to Map Paint Binders In Situ on Illuminated
Ther. 2004, 16, 53−57. Manuscripts. Angew. Chem., Int. Ed. 2012, 51, 5607−5610.
(6) Fischer, C.; Kakoulli, I. Multispectral and hyperspectral imaging (24) Dooley, K. A.; Lomax, S.; Zeibel, J. G.; Miliani, C.; Ricciardi, P.;
technologies in conservation: current research and potential Hoenigswald, A.; Loew, L.; Delaney, J. K. Mapping of egg yolk and
applications. Rev. Conserv. 2006, 7, 3−16. animal skin glue paint binders in Early Renaissance paintings using
(7) Bacci, M.; Cappellini, V.; Carla’, R. Diffuse reflectance near infrared reflectance imaging spectroscopy. Analyst 2013, 138,
spectroscopy: an application to the analysis of art works. J. Photochem. 4838−4848.
Photobiol., B 1987, 1, 132−133. (25) Kim, S. J.; Zhuo, S.; Deng, F.; Fu, C. W.; Brown, M. S.
(8) Bacci, M.; Baldini, F.; Carlà, R.; Linari, R. A Color Analysis of the Interactive visualization of hyperspectral images of historical docu-
Brancacci Chapel Frescoes. Appl. Spectrosc. 1991, 45, 26−31. ments. IEEE Transactions on Visualization and Computer Graphics.
(9) Bacci, M.; Picollo, M. Non-destructive detection of Co(II) in
2010, 16, 1441−1448.
paintings and glasses. Stud. Conserv. 1996, 41, 136−144.
(26) Conover, D. M.; Delaney, J. K.; Loew, M. H. Automatic
(10) Leona, M.; Winter, J. Fiber optics reflectance spectroscopy: a
registration and mosaicking of technical images of Old Master
unique tool for the investigation of Japanese paintings. Stud. Conserv.
paintings. Appl. Phys. A: Mater. Sci. Process. 2015, 119, 1567−1575.
2001, 46, 153−162.
(27) Conover, D. Fusion of Reflectance and X-ray Fluorescence
(11) Delaney, J. K.; Ricciardi, P.; Glinsman, L. D.; Facini, M.;
Thoury, M.; Palmer, M.; Rie, E. R. D. L. Use of Imaging Spectroscopy, Imaging Spectroscopy Data For The Identification of Artists’
Fiber Optic Reflectance Spectroscopy and X-ray Fluorescence to Map Materials. Ph.D. Thesis, George Washington University, Washington,
and Identify Pigments in Illuminated Manuscripts. Stud. Conserv. 2014, DC, 8/31/15.
59, 91−101. (28) Clark, R. N.; King, T. V. V; Klejwa, M.; Swayze, G.; Vergo, N. J.
(12) Kubik, M. Hyperspectral imaging: a new technique for the non- High spectral resolution reflectance spectroscopy of minerals. J.
invasive study of artworks. In Physical techniques in the study of art, Geophys. Res. 1990, 95, 12653−12680.
archaeology and cultural heritage; Creagh, D., Bradley, D., Eds.; Elsevier (29) Fabian, J.; Hartmann, H. Light Absorption of Organic Colorants:
Science: Amsterdam, 2007; Vol. 2, Chapter 5, pp 199−271. Theoretical Treatment and Empirical Rules; Springer-Verlag, New York,
(13) Balas, C.; Papadakis, V.; Papadakis, N.; Papadakis, A.; 1980.
Vazgiouraki, E.; Themelis, G. A novel hyper-spectral imaging apparatus (30) Jackall, Y.; Delaney, J. K.; Swicklik, M. Portrait of a Woman with
for the non-destructive analysis of objects of artistic and historic value. a Book: A “newly discovered” fantasy figure by Jean- Honoré
J. Cult. Herit. 2003, 4, 330−337. Fragonard at the National Gallery of Art, Washington. Burlington
(14) Delaney, J. K.; Zeibel, J. G.; Thoury, M.; Litteton, R.; Morales, Magazine 2015, 157, 1345.
K. M.; Palmer, M.; René de la Rie, E. Visible and Infrared Reflectance (31) Daffara, C.; Pampaloni, E.; Pezzati, L.; Barucci, M.; Fontana, R.
Imaging Spectroscopy of Paintings: Pigment Mapping and Improved Acc. Chem. Res. 2010, 43, 847−856.
Infrared Reflectography. In O3A: Optics for Arts, Architecture, and (32) Thomson, G. The museum environment, Second ed.: Butter-
Archaeology II. Proc. SPIE 7391; Pezzati, L., Salimbeni, R., Eds.; SPIE: worth- Einhemann Elsevier: London-Boston-Singapore-Sydney-Tor-
Munich, 2009; pp 739103−739103-8. onto-Wellington, 2013; Chapter 1.
(15) Casini, A.; Bacci, M.; Cucci, C.; Lotti, F.; Porcinai, S.; Picollo, (33) Frey, F.; Heller, D.; Kushel, D.; Vitale, T.; Weaver, G. AIC Guide
M.; Radicati, B.; Poggesi, M.; Stefani, L. Fiber optic reflectance to Digital Photography and Conservation Documentation; Warda, J., Ed.;
spectroscopy and hyper-spectral image spectroscopy: two integrated American Institute for Conservation of Historic and Artistic Works:
techniques for the study of the Madonna dei Fusi. In Optical Methods Washington, D.C., 2008.

2078 DOI: 10.1021/acs.accounts.6b00048

Acc. Chem. Res. 2016, 49, 2070−2079
Accounts of Chemical Research Article

(34) Bellucci, R.; Frosinini, C. The Madonna of the Yarnwinder: a

case study of non-invasive technique for art historians and
conservators. In Optical Methods for Arts and Archaeology. Proc. SPIE
5857; Pezzati, L., Salimbeni, R., Eds.; SPIE: Munich, Germany, 2005;
pp 58570K−58570K-4.
(35) Cucci, C.; Casini, A.; Picollo, M.; Poggesi, M.; Stefani, L. Open
issues in hyperspectral imaging for diagnostics on paintings: when high
spectral and spatial resolution turns into data redundancy. In O3A:
Optics for Arts, Architecture, and Archaeology II. Proc. SPIE 8084;
Pezzati, L., Salimbeni, R. Eds.; SPIE: Munich, Germany, 2011; pp
80848.1−80848.10.
(36) Cucci, C.; Casini, A.; Picollo, M.; Stefani, L. Extending Hyper-
Spectral Imaging from Vis to NIR spectral regions: a novel scanner for
the in-depth analysis of polychrome surfaces. In O3A: Optics for Arts,
Architecture, and Archaeology IV. Proc. SPIE 8790; Pezzati, L.,
Targowski, P., Eds.; SPIE: Munich, Germany, 2013; pp 8790.1−
8790.9.
(37) Burger, W.; Burge, M. J. Colorimetric Color Spaces. In Principles
of Digital Image Processing. Core Algorithms; Mackie, I., Ed.; Springer:
London, 2009; Chapter 6.
(38) Berns, R. S. The Science of Digitizing Paintings for Color-
Accurate Image Archives: A Review. J. Imaging Sci. Technol. 2001, 45,
305−325.
(39) CIE. 15:2004 Technical Report. Colorimetry, 3rd ed.;
Commission International de I’Eclairage (CIE), Central Bureau of
the CIE: Vienna.
(40) Marcus, R. T. The measurement of color. In Color for science, art
and technology; Nassau, K., Ed.; Elsevier: Amsterdam, 1998; pp 31−96.
(41) Martinez, K.; Cupitt, J.; Saunders, D.; Pillay, R. Ten Years of Art
Imaging Research. Proc. IEEE 2002, 90, 28−41.
(42) Buzzegoli, E.; Bellucci, R. Michelangelo’s ’Doni Tondo’
investigated with non-invasive analytical techniques. In Studying Old
Master Paintings: Technology and Practice. The National Gallery
Technical Bulletin 30th Anniversary Conference Postprints; Spring, M.,
Howard, H., Eds.; Archetype Publications: London, 2011; pp 52−58.
(43) Cucci, C.; Picollo, M.; Chiarantini, L.; Sereni, B. Hyperspectral
remote sensing techniques applied to the noninvasive investigation of
mural paintings: a feasibility study carried out on a wall painting by
Beato Angelico in Florence. In Optics for Arts, Architecture, and
Archaeology V. Proc. SPIE 9527; Pezzati, L., Targowski, P., Eds.; SPIE:
Munich, Germany, 2015; pp 95270P-1−95270P-9.
(44) Liang, H. Advances in multispectral and hyperspectral imaging
for archaeology and art conservation. Appl. Phys. A: Mater. Sci. Process.
2012, 106 (2), 309−323.
(45) Vitorino, T.; Casini, A.; Cucci, C.; Melo, M. J.; Picollo, M.;
Stefani, L. Non-invasive identification of traditional red lake pigments
in 14th-16th centuries paintings through the use of hyperspectral
imaging technique. Appl. Phys. A: Mater. Sci. Process. 2015, 121, 891−
901.
(46) Rosi, F.; Miliani, C.; Braun, B.; Harig, R.; Sali, D.; Brunetti, B.
G.; Sgamellotti, A. Noninvasive Analysis of Paintings by Mid-infrared
Hyperspectral Imaging. Angew. Chem. 2013, 125, 5366−5369.

2079 DOI: 10.1021/acs.accounts.6b00048

Acc. Chem. Res. 2016, 49, 2070−2079