CNG 483
Introduction to Computer Vision
Color
Asst. Prof. Dr. Meryem Erbilek
Slide partially based on Stanford U. CS131
METU Lecture 1 - 1
Overview of Color
• Physics of color
• Human encoding of color
• Color spaces
• White balancing
Slide partially based on Stanford U. CS131
METU Lecture 1 - 2
What is color?
• The result of interaction
between physical light in the
environment and our visual
system.
• A psychological property of our
visual experiences when we
look at objects and lights,
not a physical property of
those objects or lights.
Slide partially based on Stanford U. CS131 Slide credit: Lana Lazebnik
METU Lecture 1 - 3
Color and light
• When a beam of sunlight passes through a glass
prism, the emerging beam of light is not white but
consists instead of a continuous spectrum of colors
ranging from violet at one end to red at the other.
• White light: composed of almost equal energy in all
wavelengths of the visible spectrum.
Slide partially based on Stanford U. CS131
METU Lecture 1 - 4
Electromagnetic Spectrum
Human Luminance Sensitivity Function
Slide partially based on Stanford U. CS131 http://www.yorku.ca/eye/photopik.htm
METU Lecture 1 - 5
Visible Light
•Estimation of the wavelengths of electromagnetic
radiation emitted by a star is based on surface
temperature.
•For instance, since the surface of the sun is around
5800K, the peak of the sun’s emitted light lies in the
visible region. The Sun is the dominant source for visible-
light waves our eyes receive.
•A corona (outer-most layer of the Sun) is most easily https://sunearthday.nasa.gov/2007/multimedia/gal_007.php
seen during a total solar eclipse, since it is so faint and
the bright photosphere overwhelms it.
•As objects grow hotter, they radiate energy dominated
by shorter wavelengths, changing color. A flame on a
blow torch shifts from reddish to bluish in color as it is
adjusted to burn hotter. In the same way, the color of
stars tells scientists about their temperature.
Slide partially based on Stanford U. CS131
METU Lecture 1 - 6
Visible Light
Plank’s law for Blackbody radiation
Surface of the sun: ~5800K
Why do we see light of these wavelengths?
…because the
peak of the sun’s
emitted light lies in
the visible region
Slide partially based on Stanford U. CS131 © Stephen E. Palmer, 2002
METU Lecture 1 - 7
The Physics of Light
Any source of light can be completely described
physically by its spectrum: the amount of energy emitted
(per time unit) at each wavelength 400 - 700 nm.
88
© Stephen E. Palmer, 2002
The Physics of Light
Some examples of the spectra of light sources
Rel. power
Rel. power
Rel. power
Rel. power
99
© Stephen E. Palmer, 2002
The Physics of Light
Some examples of the reflectance spectra of surfaces
Yellow Blue Purple
% Light Reflected
Red
400 700 400 700 400 700 400 700
1010
Wavelength (nm) © Stephen E. Palmer, 2002
Interaction of light and surfaces
• Reflected color is the result of
interaction of light source
spectrum with surface
reflectance.
• Spectral radiometry
– All definitions and units are now
“per unit wavelength”
– All terms are now “spectral”
Slide partially based on Stanford U. CS131 From Foundation of Vision by Brian Wandell, Sinauer Associates, 1995
METU Lecture 1 - 11
Collects and processes information
from across the electromagnetic
spectrum.
1212
Interaction of light and surfaces
• What is the observed color of any surface under
monochromatic light (visible light of a narrow band
of wavelengths)?
https://www.youtube.com/watch?v=hd077pa-5CI
Slide partially based on Stanford U. CS131 Olafur Eliasson, Room for one color
METU Lecture 1 - 13 Slide by S. Lazebnik
James Turrell, a Californian artist, used light and color to
completely immerse people in a world where there was no
depth perception.
Slide partially based on Stanford U. CS131 James Turrell, Breathing Light
METU Lecture 1 - 14
Overview of Color
• Physics of color
• Human encoding of color
• Color spaces
• White balancing
Slide partially based on Stanford U. CS131
METU Lecture 1 - 16
Two types of light-sensitive receptors
•When we look at a scene, light first enters our eyes through
the pupil and then the retina.
•The retina is primarily composed of two types of light-
sensitive cells: rods and cones, named for their appearance
under a microscope.
Cones Rods
cone-shaped rod-shaped
less sensitive highly sensitive
operate in high light operate at night
color vision gray-scale vision
17
© Stephen E. Palmer, 2002
Rod / Cone sensitivity
The famous sock-matching problem… 18
Color perception
M L
Power
Wavelength
Rods and cones act as filters on the spectrum
• To get the output of a filter, multiply its response curve by the
spectrum, integrate over all wavelengths
– Each cone yields one number
• Q: How can we represent an entire spectrum with 3 numbers?
• A: We can’t! Most of the information is lost.
– As a result, two different spectra may appear indistinguishable
» such spectra are known as metamers 19
Slide by Steve Seitz
Spectra of some real-world surfaces
metamers
20
Standardizing color experience
• We would like to understand which spectra
produce the same color sensation in people
under similar viewing conditions
• Color matching experiments
21
Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995
Color matching experiment 1
22
Source: W. Freeman
Color matching experiment 1
p1 p2 p3
23
Source: W. Freeman
Color matching experiment 1
p1 p2 p3
24
Source: W. Freeman
Color matching experiment 1
The primary color
amounts needed for a
match
p1 p2 p3
25
Source: W. Freeman
Additive color mixing
Colors combine by
adding color spectra
Light adds to
existing black.
26
Source: W. Freeman
Color mixing
27
Source: W. Freeman
Examples of additive color systems
CRT phosphors
multiple projectors
http://www.jegsworks.com
28
http://www.crtprojectors.co.uk/
Color matching experiment 2
29
Source: W. Freeman
Color matching experiment 2
p1 p2 p3
30
Source: W. Freeman
Color matching experiment 2
p1 p2 p3
31
Source: W. Freeman
Color matching experiment 2
The primary color
We say a “negative” amounts needed for a
amount of p2 was match:
needed to make the
match, because we
added it to the test
color’s side.
p1 p2 p3
p1 p2 p3 p1 p2 p3
32
Source: W. Freeman
Subtractive color mixing
Colors combine by
multiplying color
spectra.
Pigments remove
color from incident
light (white).
33
Source: W. Freeman
Examples of subtractive color systems
• Printing on paper
• Crayons
• Photographic film
34
Trichromacy
• In color matching experiments, most people can
match any given light with three primaries
– Primaries must be independent
• For the same light and same primaries, most
people select the same weights
– Exception: color blindness
• Trichromatic color theory
– Three numbers seem to be sufficient for encoding
color
– Dates back to 18th century (Thomas Young)
Slide partially based on Stanford U. CS131
METU Lecture 1 - 35
Overview of Color
• Physics of color
• Human encoding of color
• Color spaces
• White balancing
Slide partially based on Stanford U. CS131
METU Lecture 1 - 36
Linear color spaces
• Defined by a choice of three primaries
• The coordinates of a color are given by the
weights of the primaries used to match it
mixing two lights produces mixing three lights produces
colors that lie along a straight colors that lie within the triangle
line in color space they define in color space
Slide partially based on Stanford U. CS131
METU Lecture 1 - 37
How to compute the weights of the primaries
to match any spectral signal
p1 p2 p3
• Matching functions: the amount of each primary
needed to match a monochromatic light source at
each wavelength
38
Source: W. Freeman
Linear color spaces: RGB space
• Primaries are monochromatic lights (for monitors,
they correspond to the three types of phosphors)
• Subtractive matching required for some
wavelengths
RGB primaries RGB matching functions
39
Nonlinear color spaces: HSV
• Perceptually meaningful dimensions:
Hue, Saturation, Value (Intensity)
40
Overview of Color
• Physics of color
• Human encoding of color
• Color spaces
• White balancing
Slide partially based on Stanford U. CS131
METU Lecture 1 - 41
White balance
• When looking at a picture on screen or print, we adapt to
the illuminant of the room, not to the scene in the picture!
• When the white balance is not correct, the picture will
have an unnatural color “cast”
incorrect white balance correct white balance
42
http://www.cambridgeincolour.com/tutorials/white-balance.htm
White balance
• Digital cameras:
• Automatic white balance
• White balance settings corresponding to
several common illuminants
• Custom white balance using a reference
object
• Film cameras:
• Different types of film or different filters for different
illumination conditions
43
http://www.cambridgeincolour.com/tutorials/white-balance.htm Slide: F. Durand
White balance
• Von Kries adaptation
• The von Kries coefficient law in color adaptation
describes the relationship between the illuminant
and the human visual system sensitivity.
• Multiply each channel by a gain factor to match the
appearance of a gray neutral object.
44
Slide: F. Durand
White balance
• Von Kries adaptation
• Multiply each channel by a gain factor.
• Best way: gray card
• Take a picture of a neutral object (white or gray)
• Deduce the weight of each channel
– If the object is recoded as rw, gw, bw
use weights 1/rw, 1/gw, 1/bw
45
Slide: F. Durand
White balance
• Without gray cards: we need to “guess” which
pixels correspond to white objects
• Gray world assumption
• The image average rave, gave, bave is gray
• Use weights 1/rave, 1/gave, 1/bave
• Brightest pixel assumption (non-staurated)
• Highlights usually have the color of the light source
• Use weights inversely proportional to the values of the
brightest pixels
• Gamut mapping
• Gamut: convex hull of all pixel colors in an image
• Find the transformation that matches the gamut of the image
to the gamut of a “typical” image under white light
• Use image statistics, learning techniques
46
Slide: F. Durand
Uses of color in computer vision
Color histograms for indexing and retrieval
47
Swain and Ballard, Color Indexing, IJCV 1991.
Uses of color in computer vision
Skin detection
M. Jones and J. Rehg, Statistical Color Models with
48
Application to Skin Detection, IJCV 2002. Source: S. Lazebnik
Uses of color in computer vision
Image segmentation and retrieval
C. Carson, S. Belongie, H. Greenspan, and Ji. Malik, Blobworld:
Image segmentation using Expectation-Maximization and its 49
application to image querying, ICVIS 1999. Source: S. Lazebnik
Uses of color in computer vision
Building appearance models for tracking
D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their 50
Appearance. PAMI 2007. Source: S. Lazebnik
Credits
• Most slides are mainly by Juan Carlos
Niebles and Ranjay Krishna from Stanford AI
Lab
51
