Systematic Literature Review: Library’s day

24 October 2024

Ria Derkx & Bas Cartigny

Today’s topics

▪ Choosing your review type

▪ Defining your review question and protocol

▪ Performing a systematic literature search

▪ Screening and organizing literature

▪ Presenting your findings

2
Schedule

Choosing your review type 9.00-10.15

Defining review question and protocol
Break 10.15-10.30
Systematic search 10.30-12.30
Lunch 12.30-13.15
Systematic search continued 13.15-14.15
Screening and organizing literature 14.15-15.15
Break 15.15-15.40
Screening and organizing literature continued 15.40-17.00
Presenting your findings
Choosing your review type
Defining review question and protocol

6
Learning goals

▪ You can mention different review types, each having its own characteristics,
strengths and weaknesses

▪ You know the characteristics and the steps to do a systematic review

▪ You can define a clear, explicit and focused review question

▪ You can find additional resources on systematic reviews in different

disciplines

7
Many types of review
Critical Review Literature Review
Mapping Review Rapid Review
Scoping Review Systematic Review
Umbrella Review State-of-the-Art Review
Mixed methods Realist Review
and more
Differences in:
✓Searching
✓Quality assessment
✓Synthesis: narrative, tabular, graphical
✓Analysis: conceptual, chronological, thematic, include
recommendations for future research
8
Comparing review types

Review type Searching Quality Synthesis

comprehensiveness assessment
Critical +++ - Narrative/conceptual/chronological

Scoping ++++(+) - Tabular with narrative commentary

Mapping ++++(+) - Graphical/tabular

Literature +++(+) -/+ Narrative

Rapid +++(+) -/+ Narrative/tabular

Umbrella No search for primary + Graphical/tabular with narrative

papers, but for reviews commentary
Systematic +++++ +++ Narrative

9
Source: Cornell University

10
Additional information on review types

https://onlinelibrary.wiley.com/
doi/epdf/10.1111/j.1471-
1842.2009.00848.x

11
Systematic review versus literature review

Narrative/literature review Systematic review

Review question Often broad in scope Focused research question

Protocol No protocol Protocol

Background Provides summaries of the available literature on a topic

Objectives May or may not be identified Clear objectives are identified

Inclusion & Exclusion criteria Not specified Criteria stated before review is
conducted
Search strategy Not explicitly stated Comprehensive search
conducted in a systematic
way. Reported, including list of
sources.
Systematic review versus literature review

Narrative/literature review Systematic review

Selecting articles Not described Clear and explicit. First on

title/abstract, then on full text

Evaluating articles May or may not be included. Based on pre-defined quality

Not described criteria

Results and data synthesis Conclusions are more Clear summaries based on
qualitative and may not be high quality evidence.
based on study quality Recommendations for practice
or to address knowledge gaps

Discussion Written by an expert or group of experts with a detailed and

well-grounded knowledge of the issues
Definition

A Systematic Review is a review of a clearly formulated question that

uses systematic and reproducible methods to identify, select and

critically appraise all relevant research, and to collect and analyse

data from the studies that are included in the review. A systematic
review can be either quantitative or qualitative.
Identify all relevant research
Using various sources

Use multiple sources to find your literature!

16
Content overlap of databases in nursing and health sciences

17
Sources for finding literature in a systematic review

▪ Bibliographic databases
▪ Supplemented by:
● Grey literature
● Registries
● Reference lists, citations, related articles
● Table of contents of specific journals
● Preprint servers, e.g. ArXiv, SocArXiv
● AI tools
● Ask an expert
● Google Scholar
●…
18
19
How to be systematic in these supplemental searches?

Report on what you did

✓ Which sources did

you search, e.g.
which websites?
✓ Include links

✓ When did you

search?

✓ How did you search?

Which keywords?
https://systematicreviewsjournal-biomedcentral- ✓ Report on use of AI
com.ezproxy.library.wur.nl/counter/pdf/10.1186/s13643-016-0371-9.pdf tools and how you
used them
20
Creating a good search strategy is the foundation of your
systematic review, it is a matter of ‘trial and error’ until you are
happy with the results. This takes time!

Try Evaluate

Improve

Until you are

21
22
23
Systematic review

▪ Systematic
▪ Thorough
▪ Transparent
▪ Repeatable
▪ Unbiased

http://en.wikipedia.org/wiki/Systematic_review
 Percentage of initially identified papers included in a systematic
review
35
73% Lowest number of initially identified publications: 30
Percentage of systematic reviews

30
Highest number of initially identified publications: 115,715
25

20
“Indeed, before the subject could be set in a clear and proper light, it
was necessary to remove a great deal of rubbish”
15 (James Lind, 1747, first controlled clinical trial)

10

0
1-2
2-3
3-4
4-5
5-6
6-7
7-8
8-9
<1

9-10

18-19
10-11
11-12
12-13
13-14
14-15
15-16
16-17
17-18

19-20
20-25
25-30
30-35
35-40
40-45
45-50
50-55
55-60
60-65
65-70
70-75
75-80
25
 Numbers of studies included in systematic reviews at WUR
25

60%
Percentage of systematic reviews

20

15

10

26
 Steps in a systematic review

http://en.wikipedia.org/wiki/Systematic_review
Formulation of review question

Review question has to be:

- Clear
- Well-defined
- Focused

https://www.dreamstime.com/
- Specific
- Appropriate
- Manageable
- Relevant to the outcomes

Never start a systematic review until a clear question can be framed

30
Tools to identify key elements for your review question

P (population/patient/problem studied)
I (intervention or treatment)
C (comparison/alternative, control)
O (outcome/expected result)
(S, T, C) (study type, time frame, context)

31
Example of PICO

What is the effect of animal-assisted therapy compared to music therapy in

managing aggressive behaviour in elderly people with dementia?

P elderly people with dementia

I animal assisted therapy
C music therapy
O aggressive behaviour

32
Tools to identify key elements for your review question

S (setting)(where)
P (population or perspective)(for whom)
I (intervention)(what)
C (comparison) (what else)
E (evaluation) (what result)

https://libguides.nus.edu.sg/
33
Example of SPICE

What are the coping skills of parents of children with

autism undergoing behavioural therapy in schools?
S schools
P parents of children with autism

https://libguides.nus.edu.sg/
I behavioural therapy
C none
E coping skills

34
Tools to identify key elements for your review question

S (sample)
P (phenomenon of ...
I (... interest)
D (design)(e.g. questionnaire, focus group, interviews)
E (evaluation)
R (research type)(e.g. qualitative, mixed method)

https://libguides.hull.ac.uk/
35
Example of SPIDER

What are young parents’ experiences of

attending antenatal education
S sample young parents
P phenomenon antenatal...
I of interest ...education
D design questionnaire

https://libguides.hull.ac.uk/
E evaluation experiences
R research type qualitative

36
Scoping

▪ Preliminary testing of search terms

● Helps you to identify key elements/concepts
● Identify additional terminology
▪ Finding out what types of relevant information exist
▪ Estimate the size of the body of evidence

Very useful before you define your final review question!

Skorupskas on Unsplash
Photo by Paul
37
Development of protocol

Define:

1. Your information need and aims

2. Eligibility criteria (inclusion/exclusion)
3. Your literature searches: resources, queries (systematic search)
4. Your management of records and data (screening and organising literature)
5. Your selection/screening, including your selection/quality criteria (screening
and organising literature)
6. Your data extraction (coding)(screening and organising literature)

38
Decide on the inclusion/exclusion criteria

Exclude on:

Date Language
Geographic location Population
Intervention Type of study
Topic Study design
Setting Type of document ....

Be careful: exclusion may increase bias!

BIAS can be assessed by tools like ROBIS, AMSTAR2
Examples of inclusion/exclusion criteria

Systematic review on diets of mammals in agricultural habitats in Europe

Included Excluded

Studies in European countries Studies in non-European countries

Publications in English Publications in other languages than English

Ecological studies Non-ecological studies e.g. laboratory studies

Mammals other than marine, livestock Marine species, livestock, other domestic species
and other domestic species

Agricultural habitats Non-agricultural habitats, e.g. forests, aquatic

environments, mountains
Studies on diets Studies on species densities and population size
PRISMA-P

Preferred reporting items for systematic review and meta-analysis protocols

(PRISMA-P) 2020 statement

https://www.jclinepi.com/article/S0895-4356(21)00073-1/pdf
43
44
Register systematic review protocol

45
Use available resources - centres

● Cochrane (medical sciences)

● The Joanna Briggs Institute (medical sciences)
● Prospero (health, social care)
● CRD (Centre for Reviews and Dissemination)(health, social care)
● Campbell (social sciences)
● EPPI (social sciences, educational sciences, sport, crime)
● SYRCLE (laboratory animal studies)
● CEE (collaboration of Environmental Evidence)(environmental sciences)
● CEBC (Centre for Evidence Based Conservation)(conservation and
environmental management)

46
Available in WUR library

47
Available in WUR library

48
 Use available resources – manuals/tutorials

▪ EFSA (European Food Safety Authority)

▪ Joanna Briggs Institute

▪ SYRCLE

▪ Yale (following Cochrane methodology)

▪ IFIS (International Food Information Services)

49
Exercise 1: review question and aim

▪ Write down
● your review question
● aim of your Systematic Review

▪ Do some scoping.

Need to adapt your review question?

For more info: ‘Orientation on topic’ and ‘Focus your topic’ in the module
‘Orientate on a topic and prepare a search’ on the library’s webpage with e-
learning modules.
Performing a systematic search: Selecting the right resources
Learning goal

▪ You can describe different types of databases.

▪ You can select an appropriate database for a specific purpose.

▪ You can describe what grey literature is.

▪ You can search and find grey literature.

What resources do you already use?
 • No reproducibility

• No quality check

• No systematic search

Do use Google Scholar to find a title you already know!

Elsevier, Public domain, via Wikimedia Commons

• Reproducible

• Quality check

• Systematic search
Types of databases

Multidisciplinary Elsevier, Public domain, via Wikimedia Commons

Subject specific
Sensitivity of models on leaching of pesticides to ground water

WoS Scopus CAB SciFinder

Total records 144 157 115 145

unique records 60 73 48 59
Off-campus access

Option 1 – Log in on library website

(library.wur.eu)

Option 2 – Install browser plugin

(https://www.leanlibrary.com/download/)

https://www.wur.nl/en/Library/FAQ-Library/FAQ-display/Off-
campus-access-to-online-resources.htm
Let’s practice!

Exercise: Find and open the databases CAB Abstracts & Zoological record
Grey literature
Grey literature

▪ Publications that have no commercial publisher e.g.:

● Reports
● Theses
● Conference proceedings
● Policy documents
● Pre-prints
● Etc

▪ Not indexed by major databases

Repositories containing grey literature

▪ OpenGrey 0,7 million

▪ OAIster 50 million
▪ BASE 240 million
▪ Science.gov 200 million US
▪ NARCIS 2,4 million Dutch
▪ IDEAS 4,1 million, Economics
▪ EconPapers 4,1 million Economics
Websites containing grey literature

▪ International organisations: WHO, FAO

▪ National organisations: US FDA, NVWA
▪ Government websites
▪ Corporate organisations
▪ Research institutes/universities: Staff Publications
(Research@WUR), ProdINRA, London School of
Economics and Political Science
▪ Research networks for systematic reviews: Campbell
Collaboration, EPPI Centre
Grey literature in Google

"urban agriculture"|"urban farming" site:.org OR site:.gov

OR site:.edu ext:pdf

"urban agriculture"|"urban farming" site:.org OR site:.gov

OR site:.edu ext:pdf 2015...2022

67
Learning goals

▪ You can describe different types of databases.

▪ You can select a database for a specific purpose.

▪ You can describe what grey literature is.

▪ You can search and find grey literature.

Exercise 2: choosing resources (30 min)

▪ What source types are you looking for? (e.g. peer reviewed articles, certain
types of grey literature, etc.)

▪ Find 4 relevant databases or other sources to find these source types.

For more info: ‘Choose your resources’ in the module ‘Orientate on a topic and
prepare a search’ on the library’s webpage with e-learning modules.
Performing a systematic search
Learning goals

▪ You can identify different search strategies

▪ You can select the most fitting search strategy for a certain purpose

▪ You can execute the search strategies discussed

 What search strategies do you already use?

a. I type some keywords into Google.

b. I ask other students/teachers/family/friends/random people where I can find the

information.

c. I thoroughly prepare a search and execute it as described in my search protocol.

d. I look at references in information sources I already have.

e. I use generative AI tools to find information.

72
1. Simple search - quick and dirty

2. Systematic search - building a complex search

3. Following a thread - snowball method

Simple search – quick and dirty

I type some words in a search box and look at the results

Simple search – example
Simple search – applications

▪ Orientation on a topic

▪ Finding search terms

▪ Getting a feeling for the amount of

research available on a topic

▪ Testing different databases

Systematic search – building a complex search

“I thoroughly prepare a search and execute it as described in my search protocol”

 1. Formulate a literature research question

2. Identify key concepts

3. Find search terms

4. Build a search query

Steps for a
systematic search 5. Execute search

6. Improve and repeat

What is the effect of artificial night lighting on
the behaviour of Parus major?

Photo of great tit by Ian Kirk, CC BY 2.0

▪ Key concepts:
● Are important things in your research question

● Are directly derived from your research question

● Are specific for your topic

Photo of great tit by Ian Kirk, CC BY 2.0

 Elephant

Mammal

Loxodonta
Africana
Systematic search

▪ Find:
● Synonyms/antonyms
● Examples of the thing (e.g. dairy -> milk)
● Singular/plural
● Word variations (farm, farmer, farming)
● UK/US spelling
● Scientific names for organisms

Wikipedia
CC BY-SA 3.0
artificial night lighting behaviour Parus major

artificial light at night behavior Great tit

nocturnal light singing ....

nocturnal illumination sleeping

.... ....
TITLE-ABS-KEY(africa* or “africa south of the sahara*” or atlas or
igboland or maghreb or maghrib or maputaland or mayombe or
nile or nubia* or sahara* or sahel or sahrawi or sub-sahara* or
subsahara* or gariep or incomaputo or incomati or juba or
shabelli or komati or kunene or limpopo or okavango or “orange
river” or ruvuma or “lake Victoria” or volta or zambezi or algeria*
This is aisland”
or angol* or “ascension searchorquery
benin*covering 1
or botswan* or “burkina
fas*” or burund*concept
or cameroon*
(Africa)orofcameroun* or “cabo verder” or
a systematic
“cape verd*” or capeverd* or “central african republic*” or chad*
or comor* or congo* orliterature review!or djibout* or egypt* or
“cote d'ivoire”
eritrea* or ethiopia* or gabon* or gambia* or ghan* or guinea*
or guinea-bissau or guinebissau or guinee or “ivory coast*” or
kenya* or lesoth* or liberia* or libya* or madagasc* or malawi*
or mali* or maroc or marocco or morocc* or maurice or
mauritania* or mauriti* or mayott* or mozambiqu* or namibia*
or niger* or nigeria* or “republique centrafricain*” or reunion or
Exercise 3: Concepts and search terms (15 minutes)

a. Identify the key concepts of your research

question/topic

b. Find all relevant search terms to describe each concept,

e.g. by doing some simple searches

For more info: ‘Search strategies’, ‘Systematic search’ in the module ‘Perform a
search and evaluate your search results’ on our webpage with e-learning
modules.
 1. Formulate a literature research question

2. Identify key concepts

3. Find search terms

4. Build a search query

Steps for a
systematic search 5. Execute search

6. Improve and repeat

Building a search query

a. Boolean operators

b. Parentheses ()

Next steps differ per database

c. Phrase searching “...”/proximity operators

d. Wildcards * ? # $
● OR: between search terms of the same concept
● AND: between different concepts
● NOT: excluding certain search terms (beware)
(artificial night lighting OR nocturnal light OR nocturnal
illumination) AND (behaviour OR sing OR sleep) AND (great tit
OR parus major)
Phrase searching

Type in Scopus:

Artificial night lighting

“Artificial night lighting”

(“artificial night lighting” OR “nocturnal light” OR “nocturnal
illumination”) AND (behaviour OR sing OR sleep) AND (“great tit”
OR “parus major”)
 Phrase searching vs proximity operators

Artificial W/4 night W/4 light

“artificial night light” artificial night light

e.g. also finds artifical light at night

In this exact order next to each other

Searched completely separately
4. Build a search query – Proximity operator
4. Build a search query - Wildcards

Wildcards in Scopus:
*=any amount of characters
?=1 character

Farm*= farm, farmer, farming, etc.

Can*=Canned OR Can OR Canteen, etc.
Cat?=cats NOT cat
Wildcards and automatic searching

Type in Scopus:

Cat

Cat?

Cats

Cat OR Cats
((artificial W/4 night W/4 light*) OR “nocturnal light*” OR
“nocturnal illuminat*”) AND (behaviour OR sing* OR sleep*)
AND (“great tit” OR “parus major”))
Number of Scopus Web of CAB, Psycinfo CAB, Zool. Nat. Sci.
characters Science (EBSCOhost) Rec. (Ovid) Coll., ASFA
(ProQuest)

0 or more * * * * *
(any)

0 or 1 (zero $ # ? ?
or one)

1 (single ? ? ? #
character)
Number of Scopus Web of Science CAB, Psycinfo CAB, Nat. Sci. Coll.,
characters (EBSCOhost) Zool. ASFA
Rec. (ProQuest)
(Ovid)

singular and yes yes yes no yes

plural (not in “...”) (not in “...”) (not in “...”)

spelling yes yes yes no yes

variations (not with*) (not with*) (not with*) (not with*)
(US/UK)

phrase “...” “...” “...” default “...” (exact)

searching (loose) (exact) (exact) is phrase

Note: in most databases phrase searching turns off automatic retrieval of singular, plural and word
variants >> “behaviour change” will not find behaviour changes or behavior change(s)
(not the case in Scopus)
Use for example: “ behav* change?” in Nat Sci Coll. or “behav* change” in Scopus (do not use ?)(also
includes behavioural, behavioral)
Fields (default) Scopus Web of CAB, Psycinfo CAB, Zool. Nat. Sci. Coll.,
and operators Science (EBSCOhost) Rec. (Ovid) ASFA
(ProQuest)

Fields (default) ti, ab, key topic All fields All fields Anywhere,
incl. full text

Booleans OR/AND/A NOT/AND/OR AND/OR/NOT AND/OR AND/OR

ND NOT NOT NOT

Proximity PRE/n NEAR/x Wn ADJn P/n

W/n Nn (n-1) N/n
 Databases – Adapting your search query

(artificial night lighting OR artificial light at night OR nocturnal light OR nocturnal illumination)
AND (behaviour OR behavior OR singing OR sleeping) AND (great tit OR parus major)

Zoological record
Elsevier, Public domain, via Wikimedia
On the OVID platform
Commons

(“artificial night light*“ OR “artificial light*” OR “nocturnal (artificial night light* OR artificial light* OR nocturnal light* OR
light*” OR “nocturnal illuminat*”) AND (behavior OR sing* nocturnal illuminat*) AND (behavio?r OR sing* OR sleep*) AND
OR sleep*) AND (“great tit” OR “parus major”) (great tit? OR parus major)
Field searching

TITLE-ABS-KEY((“artificial night light*“ OR “artificial light*” OR “nocturnal

light*” OR “nocturnal illuminat*”) AND (behavior OR sing* OR sleep*) AND
(“great tit” OR “parus major”))

Vs

ALL((“artificial night light*“ OR “artificial light*” OR “nocturnal light*” OR

“nocturnal illuminat*”) AND (behavior OR sing* OR sleep*) AND (“great
tit” OR “parus major”))

102
Thesaurus

▪ Controlled terms (vocabulary)

▪ Manually added by experts of the database
▪ Include narrower terms quickly
Let’s practice - Thesaurus

Find a thesaurus term related to your topic:

Go to CAB abstracts -> Ovid platform -> search tools ->thesaurus (in drop-down menu)
Evaluate your query

First check general errors

• Typos
• Correct application of phrases, wildcards, operators,
parentheses
Evaluate your results

Too many results?

• Use less broad terms

• Use more specific terms
• Add phrases instead of single terms
• Scoping issue: Add a concept to the research question (last resort)
Evaluate your results

Too few results?

• More word variants (wildcards, truncation)

• Add more search terms
• Use less specific terms
• Scoping issue: Remove a concept from the research question (last resort)
Evaluate this!

TITLE-ABS-KEY((artificial W/4 night W/4 light*) OR “nocturnal light*” OR

“nocturnal illuminat*”) AND (behaviour OR sing* OR sleep*) AND (“great tit”
OR “parus major”))

Results: 38
Evaluating - Validation

Select 10-20 benchmark

publications

Are they included in your results?

If not, why not? Adapt query to

find them
When to stop searching?

Ideally 100% precision and 100% recall: Not realistic

▪ Precision/specificity:
how much of the records you found is relevant?
▪ Recall/sensitivity:
how much of all relevant records have you found?
Example of a real systematic review

What is the scale of aflatoxin disease burden for African countries?

Key concepts:
1. aflatoxin
2. disease
3. africa
Example – concept 1

TITLE-ABS-KEY(aflatoxi* or AFB1 or AFB2 or AFM1 or AFM2

or AFG1 or AFG2 or AFQ1)
Example – concept 2

TITLE-ABS-KEY (health* OR Aflatoxicosis OR anemia OR Neoplasm OR

(liver W/2 cancer) OR “hepatitic cancer” OR “hepatocellular carcinoma”
OR carcinogenesis OR cirrhosis OR cirrhoses OR (liver W/2 fibrosis) OR
(liver W/2 fibroses) OR edema OR oedema OR hydrops OR Dropsy OR
anasarca OR gastritis OR “immune deficiency” OR “immunologic*
deficiency syndrome” OR “growth disorder” OR “stunted growth” OR
stunting OR (growth W/2 impair*) OR infertil* OR steril* OR subfertil*
OR “sub-fertil*” OR kwashiorkor OR malnutrition OR undernutrition OR
malnourish* OR “nutrition* deficien*” OR thinness OR underweight* OR
(kidney W/2 disorder) OR (kidney W/2 disease) OR nephropath* OR
“renal disease” OR “respiratory diseases” OR (lung W/2 disease) OR
“respiration disorder” OR “respiratory disease” OR “reyes syndrome” OR
“Reye Syndrome” OR “reye johnson syndrome” OR hepatiti*)
Example – concept 3

TITLE-ABS-KEY(africa* or “africa south of the sahara*” or atlas or igboland or maghreb or

maghrib or maputaland or mayombe or nile or nubia* or sahara* or sahel or sahrawi or “sub-
sahara*” or subsahara* or gariep or incomaputo or incomati or juba or shabelli or komati or
kunene or limpopo or okavango or “orange river” or ruvuma or (lake W/1 victoria) or volta or
zambezi or algeria* or angol* or “ascension island” or benin* or botswan* or “burkina fas*” or
burund* or cameroon* or cameroun* or “cabo verder” or “cape verd*” or capeverd* or
“central african republic*” or chad* or comor* or congo* or “cote d'ivoire” or djibout* or
egypt* or eritrea* or ethiopia* or gabon* or gambia* or ghan* or guinea* or “guinea-bissau”
or “guinebissau” or guinee or “ivory coast*” or kenya* or lesoth* or liberia* or libya* or
madagasc* or malawi* or mali* or maroc or marocco or morocc* or maurice or mauritania* or
mauriti* or mayott* or mozambiqu* or namibia* or niger* or nigeria* or “republique
centrafricain*” or reunion or rhodesia* or rwand* or “saint helena” or (“sao tome” and
principe) or senegal* or seychelles or “sierra leone” or somalia* or somaliland* or “south
africa*” or “south sudan*” or sudan* or swazi* or tanzania* or tchad* or togo* or tunisia* or
ugand* or zambia* or zimbabw*)
Managing your search process

▪ Write a logbook of your progress, decisions, etc.

▪ Save your database searches (on the search platform)

▪ Copy the search history to a Notepad document (*.txt) or OneNote, Word,

etc. or create your search strategy in Word, etc. and copy this
(unformatted) to your search platform

Important, because you have to report on this!

Search strategies in this lecture

1. Simple search - quick and dirty

2. Systematic search - building a complex search

3. Following a thread - snowball method

 reference

cited by

reference interesting article

cited by

reference

related record
When to follow a thread?

▪ At the start
● to find a few relevant
documents
● to get to know your topic
▪ During search
● to find extra search terms
▪ During or after paper selection
● to validate
Learning goal

▪ You can identify different search strategies

▪ You can select the most fitting search strategy for a certain purpose

▪ You can execute the search strategies discussed in this part

Exercise 4: search query
(Exercise 4 and 5: 45 min)

a. Combine the search terms with Boolean operators, using parentheses,

wildcards, phrase searching, and proximity operators (where needed), etc.
b. Perform the search in at least one database
c. Evaluate and improve the search query

For info: ‘Search strategies’, ‘Systematic search’ in the module ‘Perform a

search and evaluate your search results’ on the Library’s webpage with e-
learning modules.
Exercise 5: following a thread
(Exercise 4 and 5: 45 min)

a. Select one or two key publications on your topic

b. Find them in Scopus or Web of Science
c. Look at the Reference list and the Cited by articles
d. Compare these with the ones you found in a previous search
e. Add some new search terms from these new articles

For info: ‘Following a thread’ in the module ‘Perform a search and evaluate your
search results’ on the Library’s webpage with e-learning modules.
 Screening and organizing literature

http://en.wikipedia.org/wiki/Systematic_review
Learning goals

▪ You can use tools for organising, screening and selecting studies in a
systematic review

124
Why organising?

▪ Inclusion/exclusion of papers
▪ Quality assessment of papers
▪ Analysing data from studies
▪ Statistics required
▪ Identify knowledge gaps

https://indelvibe.indelcorp.co.in/

125
Report on reasons for exclusion

126
Coding (analysing data from studies)

127
Coding (analysing data from studies)

128
Analysing data from studies

129
Flow scheme

130
Software for organising, screening and selecting studies

▪ EPPI-Reviewer
▪ Covidence
▪ EROS
▪ SUMARI
▪ Distiller-SR
▪ Rayyan
▪ ASReview Lab

▪ EndNote
▪ Mendeley
▪ Zotero
▪ ...
Rayyan
Image Joukje Willemsen (CC BY-NC-ND 4.0)
ASReview Lab
Reference managers

EndNote Mendeley Zotero

No. of libraries unlimited 1 1
No. of references unlimited unlimited unlimited
(recommended limit =
100,000)
Costs not free free up to 2 GB, free free up to 300 MB,
for groups < 4 need More? pay
persons. need more?
pay
No. of journal styles 6,900 8,000 9,000
Find full text yes No yes
attach pdf manually
Sharing libraries yes yes yes
Making and sharing yes yes yes
annotations in pdfs

137
 Reference managers

EndNote Mendeley Zotero

Groups better options less options better options
Importing from various sources, from various sources, from various sources,
e.g. from bibliographic e.g. from bibliographic e.g. from bibliographic
databases databases, not so databases
good with Ebsco
databases
Deduplication you can choose which merges, you do not select master version
version you want to know which version (most information)
keep and you can you keep and merge.
keep both.
no mass
mass deduplication Some duplications deduplication. Each
possible as well. automatically duplication must be
removed, others must resolved manually
be resolved manually
Easy of use Time required to get straightforward straightforward
138
familiar
EndNote

▪ Personal reference database for

● Storing references (and their PDFs)
● Managing references
● Making reference lists in MS Word

Manual
▪ EndNote 20 and EndNote Online
Basic concepts EndNote

▪ An EndNote database is called a Library

When you create an EndNote Library, a connected data folder is
created automatically
● filename.enl
● filename.Data (settings, term lists, group information, PDFs)
▪ A library consists of references (records)
▪ References consist of fields, like author, title, journal, year etc.
▪ Groups: Subsets of references
● Automatically generated
● User defined
Add references to an EndNote library

▪ Bibliographic databases: Scopus, Web of Science, PubMed, SciFinder,

databases on the platforms EBSCOhost, Ovid, Proquest
▪ Google Scholar
▪ WUR Library Search
▪ Publisher’s websites e.g. ScienceDirect, IEEE Xplore
▪ PDF
● Single
● Folder
▪ Manually
Manage an EndNote library

▪ Display Fields
● Default fields
● Custom fields
▪ Change Field content
▪ Organise library with Groups
▪ De-duplicate
▪ Find Full text
▪ Search within your library
Display fields – Default fields

▪ Default fields (up to 10 fields)

● Choose different columns

● Adapt column headings

Via Edit > Preferences > Display Fields

Display fields – Custom fields

▪ Custom fields (up to 8 customs)

● Choose different columns

● Adapt column headings

Reviewer Topic
Included/excluded Full text ordered
Reason for exclusion …
Country
Via: Edit > Preferences > Display Fields
Change field content

▪ Add name of source (database, website etc.)

▪ Add a search date
▪ Add reviewer initials
▪ Add exclusion criteria
▪ Add quality criteria

….to all selected (!!) records

Via Library > Change/Move/Copy Fields

Organising library with groups

▪ Inclusion/exclusion of publications
▪ Reason for exclusion
● Ineligible population
● Ineligible intervention
●…
▪ Name of reviewer
▪ To organize on topic
● Country/region
● Intervention
● ...
More info:
Display fields and using custom fields for
inclusion/exclusion and name of reviewer

More info:
De-duplication of search results
De-duplication of search results

▪ Depends on field settings – By default on Author, Year, Title and Reference

Type

▪ You can change field settings via Edit > Preferences > Duplicates
Change field settings for de-duplication

Edit >
Preferences
> Duplicates
 Duplicate references?

year title secondary title (journal)

author
Range of de-duplication actions to remove as many
duplicates as possible
Follow in this order
1) Author | Year | Title | Secondary Title (Journal)
2) Author | Year | Title | Pages
3) Title | Volume | Pages
4) Author | Volume | Pages
5) Year | Volume | Issue | Pages
6) Title
7) Author | Year
Steps 3-7: Be careful not to remove false duplicates!

More info:
Demo

Break
Screening and selecting studies
Screening and selecting studies

▪ First on title and abstract

▪ Then on full text
● Quality criteria

▪ Double screening by two reviewers on a sample (± 10%)

Decide on quality criteria

From: Kirk, Catal and Tekinerdogan, 2021

https://doi.org/10.1016/j.compbiomed.2021.104365
Review of titles and abstracts

Annotated style allows you to read the abstract

Review of titles and abstracts
Start of screening: all references in unfiled group

Review titles in the Unfiled group and assign them to one of the groups you
have created

Screening finished: all references assigned to groups, unfiled group 0

Searching within your library

Very useful during screening to exclude sets of

references! However, always check what you
exclude.
Find full text

▪ Only works if:

● Connected to WUR network
● Reference has:
● DOI
● WoS record Number
● PubMed Record Number
● WUR library subscribes to Online Content or is Open Access
● Publisher allows it (EBSCO, JSTOR, OpenAthens, Wiley, Elsevier have
restrictions)

Be careful: maximally 25 references in one go!!

More restrictions when working off-campus
Find full text
Demo
An organised library
An organised library
Exercise 6: Manage EndNote library and de-duplication

a. Perform a simple search in Scopus and CAB Abstracts (Ovid)/EconLit

(EBSCOhost) and import the references into a test EndNote library
b. Think of custom fields or groups that might be useful in your project and
apply them to your test library
c. Add a remark in a specific custom field and do this in one go for a
selection of references
d. De-duplicate your Library based on the following fields:
Author | Year | Title | Secondary Title (Journal)
Author | Year | Title | Pages
Present your findings

https://pubrica.com/ 171
Learning goals

▪ You can use EndNote software for creating in-text citations and reference
lists in MS Word

▪ You can create a style for copying references from EndNote to MS Excel

172
Present findings in a scientific paper/report

all invasive plants (Richardson and Rejmánek, 2011), whereas others are commercially important crops
(Kull et al., 2011). Some species are both invasive and commercially important, like A. dealbata.
Integration with MS Word
Style for in-text citations and reference lists

Earth Science Reviews:

175
Styles

▪ EndNote offers more than 6,900 bibliographic styles

● WUR Library subscribes to over 20,000 different journals
● DOAJ includes nearly 13,500 Open Access journals

▪ If there is no style available

● Check our EndNote page for styles
● Check the official EndNote styles page
● Ask your information specialist

177
Present findings in an MS Excel file
Create your style for copying references to MS excel

More info:
Demo
Optional exercise 7: Create your style (30 min)

Choose a style for copying references to MS Word and create some in-text
citations and a reference list according to this style

and/or

Create an own style for copying references to MS Excel

Background material

▪ Information literacy modules

See For students on library website

▪ EndNote
See For students on library website
Reaching your goal

From: www.SYRCLE.nl
Questions?

[email protected]
[email protected]