KEYWORD-QUERY CONCEPT EVALUATION:
QUICK GUIDE!
August 2020
The Ads Quality Team
In this task, you will first be shown a It is recommended to approach
keyword and a query: this task in two stages.
Your task is to identify whether there is some 1. Look for close variations inside the query,
part of the query that expresses the same and then:
meaning (or same intent) as the keyword.
2. If you find a close variation to the keyword,
Answering “Yes” requires that all concepts then make sure it preserves the original
mentioned in the keyword must be contained meaning of the query, in context.
somewhere in the query.
*This differs from other task types, where
there can be several ratings available to *note: Unlike other task types you may be
define the relationship between the keyword familiar with, there are only two answers
and query (for example: Close Match available to the rater here: Yes or No.
Evaluation)
This material contains information that is proprietary and confidential to Lionbridge. It cannot be
shared with third parties without Lionbridge written consent. DO NOT COPY. DO NOT DISTRIBUTE. 2
STEP 1. LOOK FOR CLOSE VARIATIONS INSIDE THE QUERY
A close variation is a part of the query which has the same meaning or intent as the keyword.
Examples:
• Using misspellings, singular and plural forms of words, abbreviations, different word forms
(example: floor and flooring)
• Using synonyms and paraphrases (example: “mortgage” and “home loan”)
• Using words that have the same search intent (“images royalty free” and “free copyright
images”
• Reordering words (example: “insurance car” and “car insurance”)
• Adding / removing function words (example: “airports in New York”, and “airports New York”)
• Adding / removing implied words (example: “all inclusive vacations aruba price” and “all
inclusive aruba price”
This material contains information that is proprietary and confidential to Lionbridge. It cannot be
shared with third parties without Lionbridge written consent. DO NOT COPY. DO NOT DISTRIBUTE. 3
STEP 2. IF YOU DO / DON’T FIND A KEYWORD CLOSE
VARIATION:
Sometimes, other words in the query might completely change the meaning of the
close variation. In such cases you should select “No”.
If you think you can answer “Yes” make sure the query preserves the original
meaning of the keyword in context. (this is very important!)
Please check the below example for this:
Keyword Query Rating Explanation
roses where to buy No By themselves, “roses” and “rose” have the same search intent. However,
rose wine in the query, the word “rose” is used to refer to rosé, a type of wine, not a
type of flower.
roses image of red Yes The query mentions the flower referenced by the keyword.
rose in a vase
This material contains information that is proprietary and confidential to Lionbridge. It cannot be
shared with third parties without Lionbridge written consent. DO NOT COPY. DO NOT DISTRIBUTE. 4
COMMON RATING SCENARIOS
1. The query contains a close variant of the keyword and adds other words that make it
more specific. (Answer: Yes)
2. The query mentions a close variant of the keyword and adds some additional related
intent. (Answer: Yes)
3. The keyword contains information which is not reflected anywhere in the
query. (Answer: No)
4. The query is more specific than the keyword because one (or several) words in the
keyword are replaced by a more specific concept. (Answer: No)
5. There is a close variation of the keyword present in the query, but the additional query
context changes its meaning. (Answer: No)
This material contains information that is proprietary and confidential to Lionbridge. It cannot be
shared with third parties without Lionbridge written consent. DO NOT COPY. DO NOT DISTRIBUTE. 5
COMMON RATING SCENARIOS
6. Ambiguous queries and keywords
a) Start by clicking on the keyword or query, researching each.
b) If the keyword or query is unclear, has many interpretations, expresses only partial
concepts, or includes non-sensical or ungrammatical text that cannot be fully
interpreted, then treat as unrateable.
c) For ambiguous keywords: If a keyword has several reasonable and clear meanings
that are all equally likely, you should consider the interpretation that best matches the
query. However, try not to force a connection.
d) For ambiguous queries: If the keyword represents one particular interpretation of
a query which has several equally likely meanings, then you should consider that the
meaning of the keyword is not contained in the query, since it is unclear that the user
has the advertiser’s interpretation in mind.
e) If a keyword or a query has several reasonable interpretations and one is clearly
more plausible that the others, then you should consider only that interpretation.
This material contains information that is proprietary and confidential to Lionbridge. It cannot be
shared with third parties without Lionbridge written consent. DO NOT COPY. DO NOT DISTRIBUTE. 6
EXAMPLES
Keyword Query Rating Comment Scenario
dress lace Yes Query contains word “dress”, which has the same intent as dresses. 1
wedding The additional words make it more specific (particular type of dress).
dress
hairdresser training to Yes The query is looking for courses about the occupation mentioned in 2
become a the keyword. The meaning of the word “hairdresser” has not
hairdresser changed, and the query mentions a close variant of the keyword.
kia kia forte No Query and keyword mention two different Kia model names. The 3
sportage sport keyword contains some information which is not reflected anywhere
in the query.
boho style boho Yes The word “boho” by itself generally refers to the bohemian style, the 3
dresses word “style” is implied. Moreover, the query context preserves this
implication.
bedroom room decor No Keyword is referring to decor ideas for a specific room, while the 3
decor ideas ideas diy query does not reflect this.
This material contains information that is proprietary and confidential to Lionbridge. It cannot be
shared with third parties without Lionbridge written consent. DO NOT COPY. DO NOT DISTRIBUTE. 7
EXAMPLES
Keyword Query Rating Comment Scenario
LG TV LG OLED55C7 No The word “TV” was replaced by a specific model number. 4
Because of this, there is no group of words in the query which
have the same meaning as “LG TV” and can qualify as a close
variation.
LG television Yes Even though a specific model number has been added, the 4
OLED55C7 query words “LG television” have the same meaning as the
keyword.
kids kids sneakers No Kids sneakers are a specific type of shoes for kids, so there is no 4
shoes keyword close variation as part of the query.
kids sports Yes Query contains a close variation of the keyword, and adds other 1
shoes words that make it more specific.
children’s Yes “Children’s shoes” has the same meaning as kids shoes. 4
shoes light up
sneakers
This material contains information that is proprietary and confidential to Lionbridge. It cannot be
shared with third parties without Lionbridge written consent. DO NOT COPY. DO NOT DISTRIBUTE. 8
EXAMPLES
Keyword Query Rating Comment Scenario
wheels ford ranger steering No The query contains the same car model as the keyword, 5
ford wheel locked but is looking for information about its steering wheel and
ranger does not mention “wheels” as referenced in the keyword.
buy buy hdml adapter for No The word “buy” and “macbook” both appear in the query, 5
macbook macbook but the other words show that in the query these words
don’t reference the action of buying a MacBook, so their
meaning is not preserved.
pumps G.T Water Pumps-A-LOT Yes The word “pumps” can be used to refer to the mechanical 6
Plastic Water Pump in device used to move fluids
Stores near me
patent leather pump Yes The word “pumps” can also refer to a type of shoe. 6
sale near me
water pumps No The query “pumps” could refer to a type of shoe or liquid 6
pump / gas pump. Select “No” since it is not clear that the query
refers to water pumps.
This material contains information that is proprietary and confidential to Lionbridge. It cannot be
shared with third parties without Lionbridge written consent. DO NOT COPY. DO NOT DISTRIBUTE. 9
Thank you.
[email protected]
This material contains information that is proprietary and confidential to Lionbridge. It cannot be
shared with third parties without Lionbridge written consent. DO NOT COPY. DO NOT DISTRIBUTE. 10
