Gabor Filter Visualization

V. Shiv Naga Prasad∗ Justin Domke

University of Maryland University of Maryland
[email protected] [email protected]

ABSTRACT the sinusoid. Figure 1(c) shows another sinusoid whose fre-
We present a system for the visualization of an important quency is double that of the previous one; Figure 1(d) shows
signal processing technique- a Gabor Filter bank’s response the magnitude of its spectrum. Suppose we add these two si-
to an image. To do this, one must overcome the problem nusoids then we will obtain a signal as shown in Figure 1(e).
that no multi-dimensional space can be shown in a a single, Doing a spectral analysis on this would show the composi-
static graph. We use an interactive widget to change the tion of the signal - the two peaks in Figure 1(f) correspond to
visible range of the projected dimensions, and additional the component sinusoids. Fourier analysis has proven to be
graphics which summarize the responses in the projected one of the most powerful tools in signal processing. However,
dimensions. Thus, though we view this four-dimensional a key problem with Fourier analysis is that spectral features
space through 2-dimensional projections, we allow the user from different parts of the image are mixed together. Many
to understand all dimensions, not just the plane of projec- image analysis applications, e.g. object recognition, track-
tion. We found that the implemented system helped in get- ing, etc., require spatially localized features. Gabor filters
ting a better understanding of Gabor filter responses. We are a popular tool for this task of extracting spatially local-
think that use of a domain dependent interaction tool and ized spectral features [1, 2].
additional summarization graphics may be useful in a more
general Information Visualization setting. 1

0.8
80

70

0.6
60
0.4

General Terms 0.2

50

Gabor Filters, Information Visualization, High Dimensional

0 40

−0.2
30

Data −0.4
20
−0.6

10
−0.8

Keywords −1
0 20 40 60 80 100 120 140 160 180 200
0
0 10 20 30 40 50 60 70 80 90 100

Gabor Filters, Information Visualization, High Dimensional (a) (b)

1 100

Data 0.8 90

0.6 80

0.4 70

1. INTRODUCTION 0.2

0
60

50

Spatial frequencies and their orientations are important char- −0.2 40

−0.4 30

acteristics of textures in images. Figure 2 shows examples −0.6 20

of spatial textures with characteristic frequency and orien- −0.8 10

−1 0

tations. The frequency characteristics of images can be an- 0 20 40 60 80 100 120 140 160 180 200 0 10 20 30 40 50 60 70 80 90 100

alyzed using spectral decomposition methods like Fourier (c) (d)

2 120

analysis. We will illustrate spectral analysis for the simpler 1.5

100

case of 1D signals. Consider the sinusoid shown in Fig- 1

80

ure 1(a). The magnitude of its Fourier spectrum is shown 0.5

in Figure 1(b) - the peak corresponds to the frequency of 0

−0.5
60

40

The authors wish to thank Ben Shneiderman and Mustafa

−1
∗
20
−1.5

Bilgic for review and discussion about the work. −2 0

0 20 40 60 80 100 120 140 160 180 200 0 10 20 30 40 50 60 70 80 90 100

(e) (f)

Figure 1: (a) & (b) A sinusoid and its spectrum. (c)

A sinusoid with twice the frequency, (d) its spec-
trum. (e) Combination of the two sinusoids and (f )
its spectrum.

A Gabor filter bank’s response to an image consists of 4

dimensions - two of which directly correspond to the image
 1

0.8

0.6

0.4

0.2

(a) (b) 0

−0.2

Figure 2: Example of spatial frequencies in images:

−0.4

−0.6

(a) Vertical stripes - the frequencies would have hor- −0.8

izontal orientation, and (b) Curved stripes

−1
0 20 40 60 80 100 120 140 160 180 200

(a)
1

0.9

plane. Visualizing a 4D space on a screen is difficult. The 0.8

0.7

focus of our paper is to provide a good interactive interface 0.6

for this. To give a better description of the problem, we first 0.5

0.4

introduce Gabor Filters in more depth. Then we will discuss 0.3

our interface and its relation to current work in Information 0.2

0.1

Visualization. 0
0 20 40 60 80 100 120 140 160 180 200

(b)
1.1 Introduction to Gabor Filters 1

0.8

A Gabor filter is obtained by modulating a sinusoid with a 0.6

Gaussian. For the case of one dimensional (1D) signals, a

0.4

0.2

1D sinusoid is modulated with a Gaussian. This filter will 0

therefore respond to some frequency, but only in a localized −0.2

−0.4

part of the signal. This is illustrated in Figure 3. For 2D −0.6

signals such as images, consider the sinusoid shown in Fig- −0.8

−1
0 20 40 60 80 100 120 140 160 180 200

ure 4(a). By combining this with a Gaussian (Figure 4(b)),

(c)
we obtain a Gabor filter - Figure 4(c). Let g(x, y, θ, φ) be
the function defining a Gabor filter centered at the origin
Figure 3: Gabor filter composition for 1D signals:
with θ as the spatial frequency and φ as the orientation. We
(a) sinusoid, (b) a Gaussian kernel, (c) the corre-
can view Gabor filters as:
sponding Gabor filter.

x2 + y 2 has proven useful to us. The next dictate is to rank which

g(x, y, θ, φ) = exp(− ) exp(2πθi(x cos φ + y sin φ))) projections are worth considering. Here, we do not need to
σ2
(1) do this dynamically, since we are always projecting the same
4 dimensions. We can therefore predict in advance which 2-
It has been shown that σ, the standard deviation of the dimensional projections are informative. As we will discuss
Gaussian kernel depends upon the spatial frequency to mea- in detail later, these projections are the (x, y) plane, and the
sured, i.e. θ. In our case, σ = 0.65θ. Figure 5 shows 3D (θ, φ) plane. We found that if the user is given these projec-
plots of some Gabor filters and the intensity plots of their tions, the other possible projections add little. Gross et.al.
amplitudes in the image plane. See [3] for an interactive tool present an approach for generating static visualization of
to explore 2D Gabor filters. Gabor filter responses using projections [5]. However, sim-
ply showing these 2-dimensional projections statically does
The response of a Gabor filter to an image is obtained by not give a satisfactory impression of the 4-dimensional data,
a 2D convolution operation. Let I(x, y) denote the image since many 4-dimensional spaces correspond to the same
and G(x, y, θ, φ) denote the response of a Gabor filter with projections. We therefore included two techniques in our
frequency θ and orientation φ to an image at point (x, y) on visualization to give a richer impression of the data. First,
the image plane. G(.) is obtained as we designed a simple interface which allows the user to in-
teract with the projections: the user can restrict what parts
G(x, y, θ, φ) = I(p, q)g(x − p, y − q, θ, φ) dp dq (2) of the projected dimensions are visible. Second, we include
additional visualizations to give information about where in
Consider the image of a zebra shown in Figure 6(a). If we ap- the projected dimensions the data came from.
ply a Gabor filter oriented horizontally on this image then it
will give high responses wherever there are horizontal stripes 2. OUR APPROACH
present on the zebra. Figure 6(b) shows the amplitude of 2.1 One Dimensional Visualization
the response of such a horizontally oriented Gabor filter for We have devised a simple way to view the responses of Ga-
the image. bor filters in one dimension. These filter responses can be
nicely summarized in a static one-dimensional graph. This
1.2 Previous Work is interesting in its own right, and also provides an introduc-
The GRID principles [4] provide a general strategy for deal- tion to our approach for two dimensional filters.
ing with multi-dimensional data. We have used these prin-
ciples here to guide our interface design. These principles Take the response of a one-dimensional Gabor filter bank
would dictate that we begin to visualize our 4-dimensional to to be G(x, θ), where x is ’position’ and θ is frequency.
space by looking at the 2-dimensional projections, and this By creating an array indexed by x and θ and encoding the
 1

0.5

−0.5

−1
70

60

50

40

30
60
20 50
40
10 30
20
0 10
0

(a)

0.5

0
60

50

40

30

20 60
50
40
10 30
20
0 10
0

(b)

0.5

−0.5

−1
60

50

40

30

20
60
50
10 40
30
20
0 10
0

(c)
Figure 5: Example of Gabor filters with different
Figure 4: Gabor filter composition: (a) 2D sinusoid frequencies and orientations. First column shows
oriented at 30◦ with the x-axis, (b) a Gaussian ker- their 3D plots and the second one, the intensity plots
nel, (c) the corresponding Gabor filter. Notice how of their amplitude along the image plane.
the sinusoid becomes spatially localized.

problems in visualization.
strength of the response as color, we can visualize the entire
filter bank response in a single figure. For examples on syn-
Medical imaging also involves visualizing images with mul-
thetic signals, see Figures 7 and 8. For an example from a
tiple modalities simultaneously [6, 7, 8]. However, here the
real signal, see see Figure 9. Observe that for real signals, it
emphasis is on capturing the 3D human body structure. The
is very difficult to predict how a filter bank will respond to
usual approach is to stack the different image planes on top
a given signal. This is the major motivation for this work.
of one other and allowing the user to slice the across these
planes. Notice that in our case we are dealing with 4 di-
2.2 Other Possibilities mensions where only two have any explicit spatial meaning.
A straight forward extension of the 1D visualization ap- The other dimension would be created artificially by stack-
proach would be to simply show a matrix of intensity plots ing the image planes. Choosing a range of parameters would
on the screen. Each intensity plot would show the amplitude involve rotating the stack of image planes around and choos-
of Gabor filters for a particular orientation and frequency. ing a volume. It has been cited in visualization literature
The frequency could vary along the row in the matrix and that 3D rotations during visualizations are often disorient-
orientation could vary along the columns. The problem with ing as it is difficult to keep track of a frame of reference over
this approach is the lack of ability of the user to interact with the course of interaction. In our work, we have tried to get
the filter bank parameters. Typically users like to be able to the best possible interaction while confining ourselves to 2D
choose ranges of orientations and frequencies of the Gabor visualization.
filter bank and to observe the responses over the whole im-
age. Natural images rarely respond to specific frequencies or Another option would be to reuse our approach of visual-
orientation but rather exhibit a spread over these parame- izing Gabor filter responses to 1D signals. The user could
ters. Ability to dynamically choose the range of parameters be given an interface to enable him/her to slice an image
helps in better understanding of the response characteris- into a strip. Then we could apply Gabor filters of different
tics. Another issue is the pragmatics of screen real estate. frequencies along this strip and stack them as shown in Fig-
Typical images of interest in computer vision research are ure 7. However, it would be difficult to simultaneously view
of size 300 × 200. Assuming that the video screen resolu- responses for multiple orientations. Moreover, images have
tion is 1024 × 768 and we can occupy the whole screen with an inherently 2D structure - applying filters along 1D strips
the intensity plots, we can show only 3 orientations and 3 will ignore this.
scales. Even downsizing the image by half will only increase
these to 7 and 7 respectively. Downsizing further might pose Parallel coordinates are a popular approach for visualizing
 (a)

Figure 7: 1 dimensional Filter response for a syn-

thetic signal

(b)

Figure 6: (a) An image, (b) The response for Ga-

bor filter oriented horizontally - white indicates
high amplitude of response, black indicates low re-
sponse. Notice how regions of vertical stripes are
highlighted.

multi-dimensional data [9, 10]. Each dimension is plotted Figure 8: 1 dimensional Filter response for a syn-
along an axis and all axes are placed parallel to one another. thetic signal
Each data point in the high dimensional space is represented
by correspondingly joining the axes with line segments. In
our case, two of the dimensions are coordinates on the im-
way to view this data. Thus, our approach is essentially in
age plane and hence plotting them on parallel coordinates
line with the GRID principles, but applied in an unusual
might not be a good idea. (Figure 10) Star Coordinates
way. The obvious thing here, would be to observe that with
is another popular visualization tool for multi-dimensional
4 dimensions, there are 6 possible projections- why not sim-
data [11]. However, it might not useful in our case as two of
ply show them and be done with it? There are two reasons.
the dimensions have explicit spatial meaning.
The first and minor reason is that most of these 6 projec-
2.3 Our Approach tions are not meaningful. For example, a projection onto
Now, for a two dimensional Gabor filter bank, the situation the (y, φ) plane is difficult to intepret. There are really only
is much more difficult. Take the response for such a bank two projections with natural interpretations: onto the (x, y)
to be G(x, y, θ, φ), where (x, y) is the position of the filter plane, and onto the (θ, φ) plane.
relative to the input signal, θ is the frequency of the filter,
and φ is the orientation of the filter. It is clear that no static The second and more important reason, is that we have too
presentation will allow us to view the response of the entire much data for full projections. If we simply project the data
filter bank. Here we adopt the philosophy that, in order downwards onto the (x, y) plane, we will be able to see the
to give a users an understanding of the response of this 4 maximum filter response for each image point, but we will
dimensional filter bank, interaction is necessary. have no idea what part of the filter gave this response. We
combat this problem in two ways. First, we allow users to
We feel that 2 dimensional projections are again the best project onto this plane, but we also allow them to restrict
 1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
1 1.5 2 2.5 3 3.5 4 4.5 5

Figure 10: Parallel coordinate plot for a small

Figure 9: 1 dimensional Filter response for a real 10x10 image. The dimensions, from left to right
signal are: x, y, θ, φ, G(x, y, θ, φ (Since we need to show not
just the 4 dimensions, but the strength of each
point therein, in some sense we are visualizing a
what portions of the other dimensions are projected. Sec- 5-dimensional space.) Notice that even for this tiny
ondly, we use additional plots to show where in the extra image, Parallel Coordinates are very difficult to un-
dimensions the maximum value came from. derstand.
Our interface has five plots:
Though the technique was created specifically for the prob-
lem of Gabor Filter visualization, we feel that our fundamen-
1. Original image. tal idea may be of interest to the general Information Visu-
alization community, if stated more generally. When doing
2. G plot: the (x, y) projection of the Gabor filter re- a projection of high dimensional data to a two-dimensional
sponses. space, is it necessary to give up all information about the
projected dimensions? We argue that it is not necessary and
3. θ plot: this shows frequency of maximal Gabor filter provide two techniques. First, the user can interactively re-
response for different points on the image plane. strict what portions of the larger high-dimensional space are
projected. Secondly, additional plots can be used to show
4. φ plot: this shows the orientation of maximal Gabor the values of each projected dimensions. These techniques
filter response for different points on the image plane. might prove useful if applied to more general information
visualization tools, such as HCE.
5. (θ, φ) projection of the Gabor filter responses.
3. INTERFACE
In addition, we have two interaction widgets: 3.1 Original Image
This is simply a plot of the image that the Gabor filter bank
is responding to. This is important to show, because users
1. (θ, π) interaction widget: this is shown on the (θ, φ) need to compare positions on this image to positions on the
projection plane and is used to restrict the range of other plots. The zebra image is shown in Figure 6(a).
parameters of the Gabor filter bank. The user se-
lects a range in the frequency and orientation dimen-
sions of the filter bank: (θmin , φmin ) → (θmax , φmax ). 3.2 (x, y) Interaction Widget
The program then finds, for every pair (x, y), the θ The user moves a box on the image to select (xmin , ymin ) →
and φ such that G(x, y, θ, φ) is maximum, subject to (xmax , ymax ). This is shown in Figure 11. Notice that in this
θmin ≤ θ ≤ θmax , φmin ≤ φ ≤ φmax . The three fig- example, the user has selected a portion of the Zebra with
ures, namely (x, y) projection, θ plot and φ plot show stripes slightly off the horizontal. The program then finds
G, θ and φ for each point on the image. the Gabor filter parameters responding to the image signal
within this area. These are shown in the (θ, φ) projection.
2. (x, y) interaction widget: this is shown on the original
image’s (x, y) plane. It is used to select a rectangular 3.3 (θ, φ) Projection
area on the image plane: (xmin , ymin ) → (xmax , ymax ) The (θ, φ) projection is special because the orientation, φ,
Then, for each pair (θ, φ), the program finds the max- is cyclic in the range [0, π]. In order to show this clearly,
imum G(x, y, θ, φ) such that xmin ≤ x ≤ xmax , and we plot the projection in the form of a disc. Here, the fre-
ymin ≤ y ≤ ymax . The (θ, φ) projection summarizes quency, i.e. θ, varies along the radius, while φ varies along
the G(x, y, θ, φ) for each pair (θ, φ). the angular direction. The plot widget is shown in Figure 12.
 Figure 13: Interface Widget

Figure 11: (x, y) Interface Widget set by (θmin , φmin ) → (θmax , φmax ). Notice that here we
can see what parts of the image have a strong response
in the range set above, but this figure alone tells us little
The disc is discretized along the radius and angular direc- about which orientations and frequencies gave these strong
tion - in our case there were 7 quanta along θ and 10 quanta responses.
along the φ direction. Once the user has selected a range
(xmin , ymin ) → (xmax , ymax ), the program finds, for each
pair (θ, φ), the maximum response, G(x, y, θ, φ), such that
xmin ≤ x ≤ xmax , and ymin ≤ y ≤ ymax . The maximum
response is then plotted onto the (θ, φ) Projection - higher
the response, brighter the intensity of the plot. Observe
that the strongest filter responses, in this case, are medium-
frequency, and slightly off the horizontal- this corresponds
naturally to the zebra. We found that in this projection,
any windows attempting to summarize the range of x and
y which yielded the maximum response were not helpful, as
user is generally examining a small range.

Figure 14: G plot

3.6 φ plot
The plot of the φ found for each (x, y) such that G(x, y, θ, φ)
is maximum is shown in Figure Figure 15. This is not a
Figure 12: (θ, φ) Projection projection per-se, but rather tells us about what parts of
the projected dimension φ yielded the strongest response.
Rather than coding orientation as an intensity or color, we
have chosen to plot small lines with the same orientation as
3.4 (θ, φ) Interface Widget
φ. This is much easier to interpret, but we can only display
This interface is based upon a simple widget which allows φ for every few (x, y). (Otherwise the lines cover the entire
the users to select the range (θmin , φmin ) → (θmax , φmax ). image, and one cannot see anything at all.)
The widget is plotted on the (θ, φ) projection. It is in shape
of wedge in the (θ, φ) projection disc. By varying its radial
width, the user can choose different ranges of θ. Varying the 3.7 θ plot
angular width changes the range of φ in the Gabor filter. The plot of the θ found for each (x, y) such that G(x, y, θ, φ)
This scheme was chosen because it has a natural interpreta- is maximum is shown in Figure 16. Again, this is not exactly
tion: the angles spanned by the selected region, correspond a projection, but gives us information about how the projec-
to the orientation of the Gabor filters selected, while the tion was formed. Here, frequency is encoded as an intensity-
radius corresponds to the frequency. This is shown in Fig- darker colors correspond to higher frequencies. This encod-
ure 13. There, the user has selected orientations close to ing is not completely natural, but we were unable to find a
vertical, and medium frequencies. more intuitive way to display this.

3.5 G plot 4. USING THE INTERFACE

The plot of the G(x, y, θ, φ) itself is shown in Figure 14. The interface widgets can be used for exploring the filter
Strictly speaking this is the projection of the 4-dimensional responses. By tightly coupling the projections and auxil-
space down into the (x, y) plane, subject to the restrictions lary plots the user’s understanding of the multi-dimensional
 filters,” IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 12, no. 1, pp. 57–73, 1990.
[3] “An interactive tool for visualizing Gabor filters.”
[Online]. Available:
http://www.cs.rug.nl/ imaging/simplecell.html
[4] S. Jinwook and B. Shneiderman, “A rank-by-feature
framework for interactive exploration of
multidimensional data,” Univ. of Maryland - College
Park, Tech. Rep. HCIL-2004-31, 2004. [Online].
Available: ftp://ftp.cs.umd.edu/pub/hcil/Reports-
Abstracts-Bibliography/2004-31html/2004-31.pdf
[5] M. H. Gross and R. Koch, “Visualization of
Figure 15: φ plot multidimensional shape and texture features in laser
range data using complex-valued Gabor wavelets,”
IEEE Trans. Visualization and Computer Graphics,
vol. 1, no. 1, 1995.
[6] C. Rueden, K. W. Eliceiri, and J. G. White, “VisBio:
A computational tool for visualization of
multidimensional biological image data,” Traffic,
vol. 5, pp. 411–417, 2004.
[7] “Analyze: A software from mayo clinic for visualizing
medical data.” [Online]. Available:
http://www.mayo.edu/bir/HTMLs/robb FNTF 94.htm
[8] “3DViewNix: A software from the Univ. of Penn. for
visualizing multimodal medical images.” [Online].
Available:
Figure 16: θ plot http://mipgsun.mipg.upenn.edu/ Vnews/info/features.html
[9] A. Inselberg and B. Dimsdale, “Parallel Coordinates:
A tool for visualizing multidimensional geometry,” in
space is enhanced. The user can use the (x, y) interface wid- IEEE Symposium on Information Visualization, 1990,
get to find out the filter responses in regions of interest in pp. 361–375.
the image. He or she can then use the (θ, φ) interface wid-
get to find other regions in the image which produce similar [10] D. A. Keim, “Information visualization and visual
responses from the Gabor filter bank. data mining,” IEEE Trans. Visualization and
Computer Graphics, vol. 8, no. 1, pp. 1–8, 2002.
5. CONCLUSION [11] E. Kandogan, “Visualizing multi-dimensional clusters,
We have presented a system for the visualization of a difficult trends, and outliers using star coordinates,” in KDD
multi-dimensional space- the space of a Gabor Filter bank’s ’01: Proceedings of the seventh ACM SIGKDD
response to an image. The chief difficulty in doing this is international conference on Knowledge discovery and
that no single, static picture can show all information about data mining, 2001, pp. 107–116.
a multi-dimensional space such as this. We applied two tech-
niques to give additional information about the projections-
an interactive widget to change the visible range of the pro-
jected dimensions, and additional graphics which summarize
the responses in the projected dimensions. Thus, though
we view this four-dimensional space through 2-dimensional
projections, we allow the user to understand all dimensions,
not just the plane of projection. These techniques may be
useful in a more general Information Visualization setting-
when projecting to a lower-dimensional space, it is possible
to retain information about all dimensions.

6. REFERENCES
[1] D. J. Gabor, “Theory of communication,” IEE,
vol. 93, no. 26, pp. 429–457, 1946.

[2] A. C. Bovik, M. Clark, and W. S. Geisler,

“Multichannel texture analysis using localized spatial
Figure 17: Layout of the interface: top left corner - original image, bottom left corner - (x, y) projection of G
plot, top right corner - θ plot, bottom right corner - φ plot, and center - (θ, φ) projection of G plot. The (x, y)
and (θ, φ) interface widgets are shown in red on the top left and center plots respectively.

Figure 18: The user has selected the entire (x, y) range, and Gabor filters with nearly vertical orientations φ,
with medium or high frequency θ. Notice that in the bottom left plot, parts of the zerba with vertical edges
have a high response.
Figure 19: A different image. The user has selected the cadaver’s face area. The horizontal lines in the area
result in higher responses for Gabor filters with horizontal orientation, as show in the (θ, φ) plot in the center.
Because the user has selected high frequencies with any orientation, the edges in the image are highlighted.

Figure 20: The user has again selected all orientations φ, but now with medium to high frequencies θ