-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Description
Hello everyone, I have a local xray proxy (tproxy) running with inbound port 12345, and I set up iptables for proxying. Recently I had a requirement to build a k3s cluster. Since k3s uses CoreDNS as the default DNS, after finishing the setup I found that Pods could not use my host machine’s proxy.
When I checked using kubectl logs -n kube-system -l k8s-app=kube-dns, I saw that the upstream DNS was pointing to my wireless router (192.168.124.1:53). So I redirected back to 192.168.124.1:53, and then inside the Pod I ran nslookup www.baidu.com. It returned both IPv6 and IPv4. On the host machine, running nslookup www.baidu.com also returned IPv4 and IPv6, but the only difference was the order — inside the Pod, IPv6 came first.
Next, I used an ubuntu:22.04 Pod:
kubectl run dns-test --image=ubuntu:22.04 -it --rmInside the container, I ran apt update, but it kept hanging at:
0% [Connecting to archive.ubuntu.com (2620:2d:4000:1::101)] [Connecting to security.ubuntu.com (2620:2d:4000:1::102)]
Then it failed with:
Failed to fetch http://archive.ubuntu.com/ubuntu/dists/jammy-backports/InRelease
Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4002:1::102). - connect (101: Network is unreachable)
Cannot initiate the connection to archive.ubuntu.com:80 (2620:2d:4000:1::102).
However, if I add the iptables rule:
iptables -t mangle -A XRAY -i cni0 -j RETURNand restart the service, I find that inside the Pod I cannot ping www.google.com, but I can ping www.baidu.com. Also, inside the ubuntu:22.04 Pod, apt update works fine without errors.
If I use:
curl -x socks5://192.168.124.3:10808 www.google.comI can fetch the webpage data. I know that in this case traffic from cni0 does not go through the xray proxy.
But my requirement is: all Pod traffic, including DNS, must go through xray.
How should I set this up correctly in order to achieve my requirement?