Chapter 2: Image Databases

Prof. Mohammed KHALIL

 Outline

1. Image Databases

2. Visual features

3. Color

4. Texture

5. Shape
 Image databases
• Image Storage: Image databases store a large collection of images, often in
various formats such as JPEG, PNG, or GIF.
• Metadata: Each image is associated with metadata that provides information
about the image, such as its title, description, date, resolution, and more.
• Indexing: Images are indexed based on certain attributes, allowing for
efficient searching and retrieval. Common indexing methods include
keywords, tags, and categories.
• Content-Based Retrieval: Image databases often support content-based
retrieval, where images are searched based on their visual content rather
than just metadata.
Oracle Multimedia
Content based image retrieval system
Content based image retrieval system
Visual features
 Types of databases

• Generic databases :
• Heterogeneous content : public databases, Internet, general
archives
• Designed to handle a wide range of data types and applications.

• Specific databases:
• Homogeneous content : faces, fingerprints, currencies
• Tailored to address the needs of a particular industry, domain, or
application
 Image description

• Local description:
• It focuses on capturing details or features within a specific localized area of an object or data
point.
• It often involves analyzing specific parts or regions of an object to extract relevant information
• Global description:
• It provides a holistic overview or representation of the entire object or data point
• It aims to capture the overall characteristics or properties of the entire entity.
Similarity-based querying
 Feature-based querying

I want images containing 3% white, 73% blue, and 23% red.

 Local-based feature querying

• I want images where the blue color is located in the upper right corner.
 Example-based querying

• I want images with a texture similar to this image.

 Similarity-based querying

• Querying with multiple characteristics: I want images containing 50% red, 25%
granite texture.

• Shape-based querying: I want images containing an object with the following

shape.

• Object-based querying: I want images containing a red car.

• Metadata-based querying: I want images with a resolution of 300dpi, created in

2015.
 Challenges

• Image Scaling
 Challenges

• Brightness adjustments
 Challenges

• Partial visibility / occlusion

 Challenges

• Additional objects
 Challenges

• 3D objects
 Challenges
• Object detection

Returned images
Query
Schematic view
Detailed view
 Color

• Significant for categorization:

• finding the object class
• tracking objects.
 Color

• Hypothesis: If two images share similar colors then also their content may be
similar

• Example: “Retrieve all images showing a sunset !”

 Color

• The results are often quite good.

• But a frog doesn't always have the same color.

 Color features

• Important element of human perception

• Important for detection and differentiation of visual information

• Relatively easy to extract and compare

• It requires defining a color space

 What’s a Color?
A color is an event which occurs among three participants:
• Object
• Light source
• Observer
 Color Spaces
• Multi-dimensional spaces in which, various dimensions describe various color components

• Correspond to the perception of colored light by three independent receptors that are
stimulated at different wavelengths
 RGB Color Spaces
• Good representation of the visible light

• But poor usability of the similarity search

• No consistent change in the perception of color (un-) similarity

• Equal distances in different areas or different dimensions do not lead to the same color
similarity
 Color features

• To perform comparisons of image material we need to extract color features

first

• Each pixel of an image contains color information

• Aggregation for comparisons?

• Average color
• Color histograms
• Region Color
 Average color

• Calculate the average RGB values of all pixels.

• Normalize these values by the number of pixels

• Comparison of images by using the Euclidean distance for the average color.

• Quick and easy to calculate and compare.

 Average color

Example query
 Color histograms

• Partitioning of the color space

• Usually 256 values per axis in 24-bit color images (i.e. 2^24 colors, RGB)

• A histogram column for each color

• Height of the column corresponds to the normalized number of pixels with the
specified color in the image

• Normalization: scaling, so that the sum of the heights of histogram columns is 1

 Histogram

Better than average color

 Histogram

Example query
 Comparison of histograms

Given : histograms h1 and h2

• Minkowski distance:

• r=1: Histogram L1-norm (Manhattan distance)

• r=2: Histogram L2-norm (Euclidean distance)

• Works poorly in the case of color shifts because all columns are individually
compared.
Comparison between the histograms
Color Regions
 What’s a texture?

Textures describe the nature of typical, recurrent patterns in pictures

• An image obeying certain statistical properties.

• Similar structures, repeated over and over

• Often distributed in a random manner

 Texture classification
• Microtextures (or random textures): featuring microscopic primitives distributed randomly

• Macrotextures (or structured textures): spatially repeating patterns placed according to a

precise rule
 Why analyze texture?
• This provides information about the spatial distribution of colors by intensities in the image.

• Histogram of a region: 50% white pixels and 50% black pixels.

• Same intensity distribution, but three different textures.

• Texture is highly useful in similarity-based search.

 Challenges
• No mathematical model

• No satisfactory notion of distance between textures

• Highly dependent on scale

 Types of approaches

• Structural Approach: Consideration of the structural information of a

shape (use of data structures, graphs, pyramids). Suitable for macroscopic,
regular textures.

• Statistical Approach: Statistical parameters (of order 1 and order 2)

estimated for each pixel in the image: co-occurrence matrices,
neighborhood matrix, autocorrelation function.

• Frequency Approach: Fourier transformation (global representation of the

image) for regular and periodic textures.
 Structural approach : texture characteristics
• Repetition: Refers to periodic shapes and is often associated with regularity. A brick wall is a
repetitive shape, whereas an image of ocean water is non-repetitive (and lacks structure).

• Orientation: Refers to the presence or absence of directional textures. It detects the predominant
directions of elements in the image.

• Complexity: Refers to the descriptive complexity of the texture

 Statistical approach
• Statistical methods are based on the assumption that a texture is a realization of a two-
dimensional stochastic process possessing stationarity properties.

• Problem: What are the relevant statistics for texture perception?

• First-order statistics on properties.

• First-order and second-order statistics between individual grayscale values.

• First-order and second-order statistics between textons (structured elements).

Variance
Energy
Entropy
Contrast
Homogeneity
 Grey level features
• Moments of the first order do not consider the position of the pixels

• Periodicity poorly detectable

• Solution: Grey-level co-occurrence

• Pixel at position s has intensity q: I(s) = q

• Calculate the empirical probability distribution for the intensity change of the value m at
pixel shift with d pixels to the right
Grey-level Co-occurrence Matrix
 Co-occurrence matrix.

• Calculate the co-occurrence matrix for θ = 0° and 90° in both the non-
symmetric and symmetric cases
Non-symmetric Co-occurrence
Symmetric Co-occurrence