Models of Modern IR
Systems
Chapter Four- Part I
1
Template for Search Engine Evaluation
Task
Cover page (course title, group members name, search engines name)
Introduction (only one page)
About how and why you select the search engines for the comparison,
brief description about each search engines)
Comparison (only one page)
Use the table next page
Conclusion (only one page)
Discussion of key findings you want to emphasis
Reference
File naming convention – Team leader name with first letter of his/her
father -UG-IR-SEE
2
Example : tibebeb-UG-IR-SEE
Criteria SE1 SE2 SE3
Size of index database
Searching options
Stemming technique
Ranking approach
Similarity measure
approach
User interface
Relevance feedback
mechanism
Others
3
Objectives
Understanding retrieval process
Be familiar with the basic retrieval models
Get hands on experience ( simulate) retrieval of
items
4
Topics
Overview
Boolean/Logical Model
Vector Space Model (VSM)
5
Modeling of Modern IR Systems
Retrieval based on index terms assumes the
semantics of the documents and the user
information need can be expressed through sets of
index terms.
A central problem in IR is predicting which
documents are relevant and which are not.
Such a decision is dependent on a ranking algorithm
implemented.
6
Cont…
Ranking is an ordering of the documents retrieved
that (hopefully) reflects the relevance of the
documents to the user query.
Ranking is based on fundamental premises
regarding the notion of relevance, such as:
common sets of index terms
sharing of weighted terms
likelihood of relevance
7
Cont…
Such distinct set of premises (regarding document
relevance) leads to a distinct IR models.
Model
is an idealization or abstraction of the actual
process (here, retrieval).
It represents something that exists or is
planned in the real world and that in someway
is too complex or large for us to understand it
as it stands.
8
Cont…
A model is, in someway, simplified, or reduced in size,
scope or scale.
It helps to understand the system better.
And it is the best way, scientific way to study reality.
Thus, a model is a simplified representation of a complex
reality, usually for the purpose of understanding that
reality, and having all the features of that reality
necessary for the current task or problem.
9
Cont…
A model may be conceptual like a mathematical model,
which is full of equations, and are used to study the
properties of the process, draw conclusions, and make
predictions.
Statistical models on the other hand represent repetitive
processes, make predictions about frequencies of
interesting events, and use probability as the fundamental
tool.
Retrieval model ?
10
What is a retrieval model?
It is a model that describes the computational
process (e.g. how documents are ranked) and
human process (e.g. the information need,
interaction).
Note that how documents or indexes are stored
is implementation.
In relation to this, retrieval variables are queries,
documents, terms, relevance judgments, users,
information needs …
11
What an IR Model includes
Two elements
The retrieval mechanism: used to match query with a set
of documents
The ways in which the user’s information need can be
formulated as a query that can be searched by that
mechanism
thus a retrieval model specifies the details of
Document representation
Query representation
Retrieval function
12
Building a Model
To build a model, we need to think of first on
representations of the documents and the user
information need.
Given these representations the next step is to conceive
a framework in which they can be modeled.
This framework should also provide the idea on
constructing a ranking function.
In the Boolean model, the framework is composed of sets
of documents and the standard operations on sets
For the vector space model, the framework is composed
of a t-dimensional Vector space and standard linear
algebra operations on vectors 13
Cont…
The discussions made so far provide support for
discussing the two basic information retrieval models:
namely
Boolean retrieval models and
Vector space models (VSM)
14
Boolean/Logical model
It is a simple retrieval model, which is more of retrieval
than document representation and based on or uses set
theory and Boolean algebra.
Documents and queries are represented as sets of index
terms
Provides a framework, which is easy to grasp by a
common user of an IR system.
Attracted great attention in past years and was adopted
by many of the early commercial bibliographic systems.
15
Cont…
It is a basis for the majority of DBMS and conventional
IR systems.
It is the most common exact-match model.
Queries are logic expressions with document
features as operands that means query terms are
linked by the logical operators AND, OR and NOT.
A document is an object or a set consisting of
terms
Terms are features of the objects (documents):
And the search engine retrieves those documents
satisfying the logical constraints of the query.
16
Cont…
Example
Doc1: Information storage and retrieval
Doc 2: Expert system and information retrieval
systems
Doc 3: Information processing and management
Doc 4: Information retrieval in archives
17
Cont…
Index term Document
Information 1,2,3,4
Storage 1
Retrieval 1,2,4
System 2
Processing 3
Management 3
Archives 4
Suppose our query consists of information AND Retrieval
Which documents will be retrieved based on Boolean
model?
18
Cont…
The basic assumption is that there is a domain and
both the author of the document and the readers
belong to the same domain, at any one time you
have t of them.
whatit means is that any document in the
domain is written in these terms
19
Relevance – matching
Matching as a concept is the degree of similarity
between D and Q,
The degree of similarity determines the degree of
closeness between D and Q.
Ifthere is more sharing between query terms and
document terms, the author and the user are
talking the same thing.
Thus by taking intersection, what we call similarity
can be captured mathematically.
20
Cont…
Boolean model’s matching considers that index terms
are either present or absent in a document.
Thus, the index term weights are assumed to be all
binary.
That is, wij = {0, 1}
A query q is composed of index terms linked by the
three connectives, example
q = ka ( kb kc )
Boolean expressions represent a request to determine what
documents contain (or do not contain) a given set of key words.
A query searches a set of documents to determine
21
their content.
Boolean model (Document, Term, Weight, Matching
Document (how a document is viewed in BM)
Is an object, a set consisting of terms
That is, documents are sets of terms
Instance of an object (i.e., document or query) is created
when we assign value (concepts) to the features
Term (how a term is viewed in BM)
Terms are features of the objects (documents)
The terms come from the vocabulary of the subject
Represent documents in terms, of which together represent doc.
The terms are the things we used to describe concepts in a
particular domain
The vocabulary is growing when new terms are introduced
22
Cont…
Weight
Terms are either present or absent in documents
Thus, the index term weight variables are all binary, i.e.,
wij {0,1}
Matching
Degree of similarity between D and Q. If there is more sharing between
query terms and document terms, the author and the user are talking the
same thing
By taking intersection, what we call similarity can be captured
mathematically in Boolean model
Example
Q = (1, 1, o, 0, 1, 0, 0) d = (1, 0, 0, 1, 1, 0, 0) S(q, d) = 2
Thus, intersection
Takes operands and returns degree of commonness
Is a function that counts the number of matches
23
How do you explain the
essence of relevance in IRS
designed using Boolean model?
24
Example:
1. Query: Find all documents containing “information”
Boolean expression
Information
Result (means)
A set whose elements are all documents containing
the pattern “information”
25
Cont…
2. Query: Find all documents that do not contain
“information”
This is a query which attempts to find documents
that do not contain a particular pattern
Boolean expression (representation)
NOT information
Result
A set whose elements are all documents that
do not contain the pattern “information
26
Cont…
Most queries search for more than one term
Find all documents containing “information” and
“retrieval”
Find all documents containing “information” or
“retrieval” (or both)
Find all documents containing “information” or
“retrieval”, but not both
Each of the three queries illustrates a particular
concept that may form a Boolean expression, namely
Conjunction, Disjunction, Exclusive disjunction
27
Cont…
Boolean expressions may be formed from other
Boolean expressions to yield complex structure
Query
Find all documents containing “information”,
“retrieval” or not containing both “retrieval”
and “science”
Boolean expression
(Information
and retrieval) OR NOT (retrieval
AND science), parenthesis avoid ambiguity
28
Cont…
Each portion of a Boolean expression yields a set of
documents.
These portions are evaluated separately.
Combining the terms of Boolean expressions is simple
and done as follows
Let U represent the set of all docs in the collection
d1 and d2 represent those docs that contain patterns
p1 and p2 respectively.
29
The following list defines how to evaluate
Boolean expressions operators in terms of
the sets
U – d1 is the set of all docs not containing p1
(NOT)
d1 ∩ d2 is the set of all docs. containing both
p1 and p2 (AND)
d1 U d2 is the set of all docs. containing
either p1 or p2 (OR)
d1 U d2 – d1 ∩ d2 is the set of all docs.
Containing either p1 or p2, but not both (XOR)
30
Cont…
Thus In Boolean,
the use of AND requires that both terms that it
connects be present in the retrieved documents,
the use of OR requires that at least one of the terms
be present.
This is an inclusive use of OR, meaning that it is
acceptable for both of the terms to be present,
31
Cont…
If an exclusive use of OR is desired- one term or the
other, but not both- the construction is more
complex:
(A AND NOT B) OR (B AND NOT A)
or
(A OR B) AND NOT (A AND B)
NOT requires that the specified term be absent from
any retrieved document
32
Exercise: Consider a set of five docs and assume that
they contain the terms shown in the table
Doc. Terms
D1 Algorithm, information, retrieval
D2 Retrieval, science
D3 Algorithm, information, science
D4 Pattern, retrieval, science
D5 Science, algorithm
Find documents retrieved by the following expressions
• Information AND retrieval
• Information OR retrieval
• (Information and Retrieval) OR NOT (Retrieval and Science)
33
Solution
Information AND retrieval
{d1,d3} ∩{d1,d2,d4}={d1}
Information OR retrieval
{d1,d3} U {d1,d2,d4}={d1, d2,d3,d4}
(Information and Retrieval) OR NOT (Retrieval
and Science)
(d1) OR NOT (d4,d2)= {d1,d3,d5}
34
Advantages of the Boolean
model
Simplicity
Isstill a dominate model with the
commercial database systems
Providesa good starting point for those
new to the field
35
Limitations (Drawbacks) of the
Boolean Model
In pure Boolean, there is no good way to weight
terms for significance. (thus it does only binary
partition
Either a term is present or absent. Thus, the
user has little control over how important a
given term is to the query.
That is, Its retrieval strategy is based on
binary decision criteria.
No weighting for document terms and no
weighting for query terms
36
Cont…
That is, the significance concept is totally ignored.
The representation is only binary.
The system is not flexible to represent weight which is
said very important in IR.
Reconsideration of index weight brings us to the
vector model
Predicts that each document is either relevant or
non-relevant.
Is a simple partition - those that match the query and
those that do not?
Divides the collection into two subsets only,
retrieved and non-retrieved 37
Cont…
There is no notion of partial matching to the query
condition.
For example, let dj be a document for which
vector
dj= (0,1,0)
Document dj includes the index term kb but is
considered non- relevant to the query
ka (kb kc)
This prevents good retrieval performance.
38
Cont…
In Boolean model no ranking of the documents is
provided (absence of a grading scale)
as all documents are considered equal, no
ordering of retrieved set
Retrieved documents are generally not ranked.
All retrieved are presumed to be equally useful.
No mechanism to show the relative importance of
the different components of a query
39
Cont…
Query formulation is too difficult using the Boolean
operators.
Boolean expressions have precise semantics
Thus, it is not simple to translate an information
need into a Boolean expression.
Informationneed has to be translated into a
Boolean expression which most users find awkward.
40
Cont…
To answer sophisticated queries we need to know
more about Boolean logic.
We need also to have good knowledge of representing
queries in Boolean logic, which presumes knowledge
of the document, queries (user’s needs) and so on
As a consequence there is a
Need for trained intermediary, which create
another problem, problem of understanding.
Instead of yourself, somebody do the translation
for you on your behalf.
41
Cont…
Boolean model frequently returns either too few or too
many documents in response to a user query
As it is very difficult to precisely define users need at the
beginning
As a result of which, the Boolean model frequently
returns either too few or too many documents in
response to a user query
That is, exact matching may lead to retrieval of too few or
too many documents (main problem)
This shows very little control over the size of the output by a
particular query.
That is, the size of retrieved set can hardly be controlled.
42
Cont…
“NOT”, for instance, retrieves every document that does
not contain a specific term.
A query such as ‘NOT aardvark’ runs of retrieving virtually
the entire database
Again another point is separation between retrieved / non-
retrieved too strict that means
For q= t1 Λ t2 Λ t3, documents containing two of the
terms will be rejected as well as those containing none
Analogously for q=t1 V t2 V t3, no ordering within
retrieved documents
Generally it has poor retrieval quality and its main problem
is the inability to recognize partial matches which
frequently leads to poor performance 43
Exercise
Given the following four documents with the following
contents:
D1 = “computer information retrieval”
D2 = “computer retrieval”
D3 = “information”
D4 = “computer information”
What are the relevant documents retrieved for the
queries:
Q1 = “information retrieval”
Q2 = “information ¬computer”
44
The Boolean Model: Example
• Given the following determine documents retrieved by the Boolean
model based IR system
• Index Terms: K1, …,K8.
• Documents:
1. D1 = {K1, K2, K3, K4, K5}
2. D2 = {K1, K2, K3, K4}
3. D3 = {K2, K4, K6, K8}
4. D4 = {K1, K3, K5, K7}
5. D5 = {K4, K5, K6, K7, K8}
6. D6 = {K1, K2, K3, K4}
• Query: K1 (K2 K3)
45
• Answer: {D1, D2, D4, D6} ({D1, D2, D3, D6} {D3, D5})
= {D1, D2, D6}
46
The Boolean Model: Example
Given the following three documents, Construct Term – document matrix and
find the relevant documents retrieved by the Boolean model for given
query
• D1: “Shipment of gold damaged in a fire” • Find the relevant
• D2: “Delivery of silver arrived in a silver truck” documents for the
• D3: “Shipment of gold arrived in a truck” queries (use AND , OR)
• Query: “gold silver truck” (a)gold delivery
Use table below for the –term matrix (b)ship gold
(c)silver truck
arrive damage deliver fire gold silver ship truck
D1
D2
D3
query 47
Next
On Vector Space Model
48
