Codestin Search App

MarcAbonce · 2016-05-20T05:39:11Z

With engines Ahmia, Torch and Not Evil.

Disabled by default. Can be enabled by setting the SOCKS proxies to wherever Tor is listening and setting using_tor_proxy as True.

To avoid manually adding the timeout on each engine, you can set extra_proxy_timeout to account for Tor's (or whatever proxy used) extra time.

Should solve #413 and probably #519

GreenLunar · 2016-05-20T05:51:19Z

There is also not Evil at http://hss3uro2hsxfogfq.onion/ (was TorSearch at http://kbhpodhnfxl3clb4.onion/)

MarcAbonce · 2016-05-31T06:13:46Z

By the way, ~~htk2rvtgj6vv3ppj.onion~~ is currently running this branch.
Edit: ~~mze6ter6fchwojcr.onion~~ ~~http://jiwfbtg2kfkadjku.onion~~ http://7ea7vegvcqgdzxjz.onion

asciimoo · 2016-07-25T13:44:14Z

@a01200356 great, the only problem is that it isn't compatible with the existing non-tor configuration. It would be cool to provide both to the users. Could be a separated settings_tor.yml for the tor config.

MarcAbonce · 2017-01-07T04:27:41Z

Sorry for the delay.
I just updated the PR. But instead of having two settings files, I added the using_tor_proxy setting that should automatically set up all the other Tor specific configurations.
When set to False or commented out, as default, everything should behave exactly the same as current version on upstream.

MarcAbonce · 2017-02-04T05:06:45Z

I just fixed the issue I had with not evil's encoding.

About the last commit, I just assume that using Tor Browser's user agent would make our traffic seem more "normal" when using Tor than using a random user agent. However, this is merely an assumption on my part. I can't really tell if it actually helps or not.

ghost · 2017-02-19T12:11:35Z

Looks interesting.

dalf · 2017-03-29T19:24:28Z

searx/utils.py

    # TODO
-    return ua.format(os=choice(ua_os), version=choice(ua_versions))
+    if settings['outgoing'].get('using_tor_proxy'):
+        return tor_browser_ua


Why a specific behavior for Tor ?

I think that, when using Tor, Tor Browser's user agent would make our requests blend in better with "normal" Tor users, while using unique agents would only make our requests stand out compared to the other (non Searx) requests coming out of the same exit node.

MarcAbonce · 2017-12-27T04:31:16Z

~~With the fix in #1121, Google doesn't block Tor exit nodes any more.~~
I also reverted the Tor Browser user-agent in Tor requests, I don't think it makes any difference anyway.
Also, DuckDuckGo currently uses the normal website because the hidden service is extremely slow, which makes it reach timeout most of the time.
An updated instance running this branch is in ~~http://jiwfbtg2kfkadjku.onion~~ http://7ea7vegvcqgdzxjz.onion

asciimoo · 2020-09-28T13:38:55Z

@MarcAbonce sorry for the delayed answer. I still think that this is a useful functionality, could you please rebase it to the latest master? I'm happy to do it if you don't have capacity.

Xpath engine and results template changed to account for the fact that archive.org doesn't cache .onions, though some onion engines migth have their own cache. Disabled by default. Can be enabled by setting the SOCKS proxies to wherever Tor is listening and setting using_tor_proxy as True. Requires Tor and updating packages. To avoid manually adding the timeout on each engine, you can set extra_proxy_timeout to account for Tor's (or whatever proxy used) extra time.

MarcAbonce · 2020-10-26T02:47:10Z

I rebased to master and I also added another commit.

asciimoo

Awesome, thanks!

asciimoo · 2020-10-26T13:22:45Z

This is a great addition, we should mention it in the documentation.

dalf · 2020-10-27T19:09:14Z

Note about the plugin, it can't be disabled, I guess on purpose :

searx/searx/plugins/__init__.py

Lines 179 to 188 in db703a0

    
           if 'enabled_plugins' in settings: 
        
               for plugin in plugins: 
        
                   if plugin.name in settings['enabled_plugins']: 
        
                       plugin.default_on = True 
        
                   else: 
        
                       plugin.default_on = False 
        
           # load tor specific plugins 
        
           if settings['outgoing'].get('using_tor_proxy'): 
        
               plugins.register(ahmia_filter)

The plugin is loaded after enabled_plugins parsing and in the preferences (in the oscar theme), it is hidden:

searx/searx/templates/oscar/preferences.html

Line 261 in db703a0

{% if plugin.preference_section != 'onions' %}

MarcAbonce · 2020-10-28T05:51:44Z

Yeah, maybe it would be better to extend the new preferences lock to include plugins and use that instead?

Perflyst · 2020-12-15T21:30:35Z

Looks like if a proxy is configured the proxy is used for all search engines and not only for onion search engines. Is there an option to use the proxy (in this specific use-case the Tor socks proxy) only for onion search engines?

dalf · 2020-12-15T22:09:11Z

You can specify a proxy per engine.
See https://searx.github.io/searx/admin/settings.html#engine-settings

So you have to copy / paste the proxy section for each onion engine
or you can use YAML anchor ( https://en.wikipedia.org/wiki/YAML#Advanced_components )

asciimoo force-pushed the master branch from a94ff7b to 1ee2a3d Compare June 26, 2016 12:34

dalf added enhancement core engine labels Jul 19, 2016

MarcAbonce force-pushed the onions branch from d3923d5 to bad849c Compare January 7, 2017 01:24

MarcAbonce force-pushed the onions branch from bad849c to 8f99379 Compare January 9, 2017 23:30

MarcAbonce force-pushed the onions branch from 8f99379 to b320bfc Compare February 4, 2017 03:43

MarcAbonce mentioned this pull request Mar 29, 2017

How to use through Tor #886

Closed

dalf reviewed Mar 29, 2017

View reviewed changes

MarcAbonce force-pushed the onions branch 5 times, most recently from 9bfeca1 to cf589bb Compare June 7, 2017 02:35

MarcAbonce force-pushed the onions branch 2 times, most recently from d403002 to ecd9827 Compare December 27, 2017 02:01

MarcAbonce mentioned this pull request Jan 22, 2018

search in darkweb with Searx #1155

Closed

MarcAbonce force-pushed the onions branch 6 times, most recently from 518b307 to 26f18a6 Compare July 4, 2019 05:45

MarcAbonce force-pushed the onions branch from 26f18a6 to 6e93240 Compare August 31, 2019 23:52

MarcAbonce force-pushed the onions branch from 6e93240 to bebe808 Compare October 20, 2019 00:15

MarcAbonce force-pushed the onions branch from bebe808 to 0feaae0 Compare November 3, 2019 23:57

MarcAbonce force-pushed the onions branch 2 times, most recently from 0315d3f to 1ec3717 Compare March 8, 2020 08:00

MarcAbonce force-pushed the onions branch from 1ec3717 to 659ced0 Compare March 31, 2020 05:04

MarcAbonce mentioned this pull request Sep 16, 2020

Service's IP address leak, even if it is a hidden service #2197

Open

MarcAbonce and others added 2 commits October 25, 2020 17:59

add Ahmia filter plugin for onion results

32957cd

MarcAbonce force-pushed the onions branch from 659ced0 to 32957cd Compare October 26, 2020 02:00

asciimoo approved these changes Oct 26, 2020

View reviewed changes

asciimoo merged commit db703a0 into searx:master Oct 26, 2020

demostanis mentioned this pull request Jan 27, 2021

Deep Web Search support #2455

Closed

MarcAbonce deleted the onions branch March 15, 2021 03:21

dalf mentioned this pull request Dec 22, 2021

"onions" category broken searxng/searxng#643

Closed

return42 mentioned this pull request Jan 14, 2025

[json_engine] Documentation and overhaul searxng/searxng#4179

Merged

Comments

Conversation

MarcAbonce commented May 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GreenLunar commented May 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MarcAbonce commented May 31, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asciimoo commented Jul 25, 2016

Uh oh!

MarcAbonce commented Jan 7, 2017

Uh oh!

MarcAbonce commented Feb 4, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Feb 19, 2017

Uh oh!

dalf Mar 29, 2017

Choose a reason for hiding this comment

Uh oh!

MarcAbonce Mar 30, 2017

Choose a reason for hiding this comment

Uh oh!

MarcAbonce commented Dec 27, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asciimoo commented Sep 28, 2020

Uh oh!

MarcAbonce commented Oct 26, 2020

Uh oh!

asciimoo left a comment

Choose a reason for hiding this comment

Uh oh!

asciimoo commented Oct 26, 2020

Uh oh!

dalf commented Oct 27, 2020

Uh oh!

MarcAbonce commented Oct 28, 2020

Uh oh!

Perflyst commented Dec 15, 2020

Uh oh!

dalf commented Dec 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

MarcAbonce commented May 20, 2016 •

edited

Loading

GreenLunar commented May 20, 2016 •

edited

Loading

MarcAbonce commented May 31, 2016 •

edited

Loading

MarcAbonce commented Feb 4, 2017 •

edited

Loading

MarcAbonce commented Dec 27, 2017 •

edited

Loading