using proxy + HTTPS ends with SSL session is not started yet #389
Description
Hello,
there seems to be some issue with (may be race condition of slow non-caching) http proxy when requesting HTTPS sites.
Without proxy
$ whatweb https://www.root.cz/
https://www.root.cz/ [200 OK] Cookies[iinfo-sid], Country[CZECH REPUBLIC][CZ], Email[[email protected],[email protected]], Frame, HTML5, HTTPServer[nginx/1.14.2], HttpOnly[iinfo-sid], IP[91.213.160.188], JQuery[2.1.4], Open-Graph-Protocol[article][160071904046461], PasswordField[us_pwd], Script[text/javascript], Strict-Transport-Security[max-age=63072000; includeSubDomains; preload], Title[Root.cz - informace nejen ze světa Linuxu], UncommonHeaders[x-networkmanager-status,content-security-policy,x-backend-server,referrer-policy,access-control-allow-header], X-Backend[web2 D=126273 t=1677330589429008, web3-new], nginx[1.14.2]
With proxy
$ whatweb --proxy localhost:8118 https://www.root.cz/
ERROR Opening: https://www.root.cz/ - SSL session is not started yet
Curl / wget ang others go well through the proxy (privoxy in my case)
$ curl -I --proxy localhost:8118 https://www.root.cz/ 2>&1|head -n 30
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
HTTP/1.1 200 Connection established
HTTP/2 200
server: nginx/1.14.2
date: Sat, 25 Feb 2023 13:13:42 GMT
content-type: text/html; charset=UTF-8
strict-transport-security: max-age=63072000; includeSubDomains; preload
x-networkmanager-status: online
set-cookie: iinfo-sid=gjqti1ulr6a58f2u6mtoknh5cn; expires=Tue, 07-Mar-2023 13:13:42 GMT; Max-Age=864000; path=/; domain=.root.cz; secure; HttpOnly
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: no-store, no-cache, must-revalidate
pragma: no-cache
vary: X-Requested-With
x-backend-server: web8 D=422603 t=1677330822338638
referrer-policy: no-referrer-when-downgrade
access-control-allow-header: *
x-backend-server: web3-new
Workaround - It works using proxychains:
$ proxychains whatweb https://www.root.cz/
[proxychains] config file found: /home/mambroz/.proxychains/proxychains.conf
[proxychains] preloading /usr/lib64/proxychains-ng/libproxychains4.so
[proxychains] DLL init: proxychains-ng 4.16
[proxychains] DLL init: proxychains-ng 4.16
[proxychains] Strict chain ... 127.0.0.1:8118 ... 91.213.160.188:443 ... OK
https://www.root.cz/ [200 OK] Cookies[iinfo-sid], Country[CZECH REPUBLIC][CZ], Email[[email protected],[email protected]], Frame, HTML5, HTTPServer[nginx/1.14.2], HttpOnly[iinfo-sid], IP[91.213.160.188], JQuery[2.1.4], Open-Graph-Protocol[article][160071904046461], PasswordField[us_pwd], Script[text/javascript], Strict-Transport-Security[max-age=63072000; includeSubDomains; preload], Title[Root.cz - informace nejen ze světa Linuxu], UncommonHeaders[x-networkmanager-status,content-security-policy,x-backend-server,referrer-policy,access-control-allow-header], X-Backend[web7 D=288462 t=1677330894285181, web3-new], nginx[1.14.2]
[mambroz@czcholn5025023:~] 2023-02-25 14:15: