Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ end
- [Bingbot](https://blogs.bing.com/webmaster/2012/08/31/how-to-verify-that-bingbot-is-bingbot/)
- [BLEXBot (WebMeUp)](http://webmeup-crawler.com/)
- [DataForSEO](https://dataforseo.com/dataforseo-bot)
- [DuckDuckGo bot](https://duckduckgo.com/duckduckbot)
- [DuckAssistBot](https://duckduckgo.com/duckduckgo-help-pages/results/duckassistbot)
- [DuckDuckBot](https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot)
- [Google crawlers](https://support.google.com/webmasters/answer/1061943)
- [IAS](https://integralads.com/ias-privacy-data-management/policies/site-indexing-policy/)
- [OpenAI GPTBot](https://platform.openai.com/docs/gptbot)
Expand Down
257 changes: 254 additions & 3 deletions lib/legitbot/duckduckgo.rb
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# frozen_string_literal: true

module Legitbot # :nodoc:
# https://duckduckgo.com/duckduckbot
# https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot
# rubocop:disable Metrics/ClassLength
class DuckDuckGo < BotMatch
class DuckDuckBot < BotMatch
# @fetch:url https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot
# @fetch:selector main > article ul > li > div
ip_ranges %w[
Expand Down Expand Up @@ -251,5 +251,256 @@ class DuckDuckGo < BotMatch
end
# rubocop:enable Metrics/ClassLength

rule Legitbot::DuckDuckGo, %w[DuckDuckBot]
# https://duckduckgo.com/duckduckgo-help-pages/results/duckassistbot
# rubocop:disable Metrics/ClassLength
class DuckAssistBot < BotMatch
# @fetch:url https://duckduckgo.com/duckduckgo-help-pages/results/duckassistbot
# @fetch:selector main > article ul > li > div
ip_ranges %w[
4.156.136.79
4.182.131.108
4.195.133.120
4.209.224.56
4.213.46.14
4.228.76.163
13.89.106.77
20.3.1.178
20.12.141.99
20.40.133.240
20.43.150.85
20.43.150.93
20.43.172.120
20.44.222.1
20.49.136.28
20.50.48.159
20.50.48.192
20.50.49.0
20.50.49.25
20.50.49.40
20.50.49.55
20.50.49.237
20.50.50.46
20.50.50.118
20.50.50.121
20.50.50.123
20.50.50.130
20.50.50.134
20.50.50.145
20.50.50.146
20.50.50.163
20.50.168.91
20.53.78.106
20.53.78.123
20.53.78.138
20.53.78.144
20.53.78.236
20.53.91.2
20.53.92.211
20.53.134.160
20.56.197.58
20.56.197.63
20.61.34.40
20.62.224.44
20.71.12.143
20.72.242.93
20.73.132.240
20.73.202.147
20.75.144.152
20.79.226.26
20.79.238.198
20.79.239.66
20.80.129.80
20.93.28.24
20.99.255.235
20.113.3.121
20.113.14.159
20.185.79.15
20.185.79.47
20.191.44.16
20.191.44.22
20.191.44.119
20.191.44.234
20.191.45.212
20.193.12.126
20.193.24.10
20.193.24.251
20.193.25.197
20.193.27.215
20.193.45.113
20.195.108.47
20.197.209.11
20.197.209.27
20.201.15.208
20.204.240.172
20.204.241.148
20.204.242.19
20.204.242.101
20.204.243.55
20.204.246.81
20.204.246.254
20.207.72.11
20.207.72.21
20.207.72.110
20.207.72.113
20.207.97.190
20.207.99.197
20.207.107.181
20.219.43.246
20.219.45.67
20.219.45.190
20.226.133.105
20.232.184.230
40.64.105.247
40.64.106.11
40.76.162.191
40.76.162.208
40.76.162.247
40.76.163.7
40.76.163.23
40.76.173.151
40.80.242.63
40.81.250.205
40.88.21.235
40.89.243.175
40.114.182.45
40.114.182.153
40.114.182.172
40.114.183.88
40.114.183.196
40.114.183.251
40.119.232.50
40.119.232.146
40.119.232.215
40.119.232.218
40.119.232.251
51.8.71.117
51.8.253.152
51.104.146.225
51.104.146.235
51.104.160.167
51.104.160.177
51.104.161.32
51.104.162.149
51.104.163.250
51.104.164.109
51.104.164.147
51.104.164.189
51.104.164.215
51.104.166.111
51.104.167.19
51.104.167.52
51.104.167.54
51.104.167.61
51.104.167.71
51.104.167.87
51.104.167.88
51.104.167.95
51.104.167.96
51.104.167.104
51.104.167.110
51.104.180.26
51.104.180.47
51.104.180.53
51.107.40.209
51.116.131.221
51.120.48.122
51.138.90.161
51.138.90.206
51.138.90.233
52.142.24.149
52.142.26.175
52.143.95.162
52.143.95.204
52.143.241.111
52.143.242.6
52.143.243.117
52.143.244.81
52.143.247.235
52.146.58.236
52.146.59.12
52.146.59.154
52.146.59.156
52.146.63.80
52.148.161.87
52.148.165.38
52.149.25.43
52.149.28.18
52.149.28.83
52.149.30.45
52.149.56.151
52.149.58.27
52.149.58.69
52.149.58.139
52.149.58.173
52.149.60.38
52.149.61.51
52.154.60.82
52.154.169.50
52.154.169.200
52.154.170.26
52.154.170.28
52.154.170.88
52.154.170.96
52.154.170.113
52.154.170.117
52.154.170.122
52.154.170.209
52.154.170.229
52.154.170.243
52.154.171.0
52.154.171.44
52.154.171.70
52.154.171.87
52.154.171.150
52.154.171.196
52.154.171.205
52.154.171.235
52.154.171.250
52.154.172.2
52.190.37.160
52.224.16.221
52.224.16.229
52.224.19.152
52.224.20.174
52.224.20.181
52.224.20.186
52.224.20.190
52.224.20.193
52.224.20.203
52.224.20.204
52.224.20.223
52.224.20.227
52.224.20.249
52.224.21.4
52.224.21.19
52.224.21.20
52.224.21.23
52.224.21.27
52.224.21.49
52.224.21.51
52.224.21.53
52.224.21.55
52.224.21.61
52.242.224.168
57.152.72.128
104.43.54.127
104.43.55.116
104.43.55.117
104.43.55.166
104.43.55.167
108.141.83.74
172.169.17.165
191.233.3.197
191.233.3.202
191.234.216.4
191.234.216.178
191.235.201.214
191.235.202.38
191.235.202.48
]
end
# rubocop:enable Metrics/ClassLength

rule Legitbot::DuckDuckBot, %w[DuckDuckBot]
rule Legitbot::DuckAssistBot, %w[DuckAssistBot]
end