OpenVPN on MrVM LES nat VPS : no forwarding
Hello,
I installed OpenVPN using https://github.com/angristan/openvpn-install and it seems to work ok as I am able to establish a connection, ping the VPS and even ssh into it using the VPN private ip. The problem is that the client does not have full Internet access. Ping to 8.8.8.8 does not succeed, using tcpdump I see the ICMP packets arriving on the VPS but I guess they are not correctly forwarded.
I tried every iptables rule I could find here and there, either MASQUERADE or SNAT but with no success (iptables is not complaining but it does not work). net.ipv4.ip_forward is obviously set to 1.
I am clueless, if anyone has an idea I would be really grateful.
OpenVPN client log :
sudo openvpn --config tony.ovpn
Sun May 3 23:10:47 2020 Unrecognized option or missing or extra parameter(s) in tony.ovpn:19: block-outside-dns (2.4.7)
Sun May 3 23:10:47 2020 OpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Sep 5 2019
Sun May 3 23:10:47 2020 library versions: OpenSSL 1.1.1f 31 Mar 2020, LZO 2.10
Sun May 3 23:10:47 2020 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Sun May 3 23:10:47 2020 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
Sun May 3 23:10:47 2020 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Sun May 3 23:10:47 2020 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
Sun May 3 23:10:47 2020 TCP/UDP: Preserving recently used remote address: [AF_INET]213.239.XXX.YYY:19001
Sun May 3 23:10:47 2020 Socket Buffers: R=[212992->212992] S=[212992->212992]
Sun May 3 23:10:47 2020 UDP link local: (not bound)
Sun May 3 23:10:47 2020 UDP link remote: [AF_INET]213.239.XXX.YYY:19001
Sun May 3 23:10:47 2020 TLS: Initial packet from [AF_INET]213.239.XXX.YYY:19001, sid=8bc6ec6a f365ad3c
Sun May 3 23:10:47 2020 VERIFY OK: depth=1, CN=cn_x07KORhkeMrqJgrY
Sun May 3 23:10:47 2020 VERIFY KU OK
Sun May 3 23:10:47 2020 Validating certificate extended key usage
Sun May 3 23:10:47 2020 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
Sun May 3 23:10:47 2020 VERIFY EKU OK
Sun May 3 23:10:47 2020 VERIFY X509NAME OK: CN=server_NoElFKC3RsPyQtKK
Sun May 3 23:10:47 2020 VERIFY OK: depth=0, CN=server_NoElFKC3RsPyQtKK
Sun May 3 23:10:47 2020 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 256 bit EC, curve: prime256v1
Sun May 3 23:10:47 2020 [server_NoElFKC3RsPyQtKK] Peer Connection Initiated with [AF_INET]213.239.XXX.YYY:19001
Sun May 3 23:10:48 2020 SENT CONTROL [server_NoElFKC3RsPyQtKK]: 'PUSH_REQUEST' (status=1)
Sun May 3 23:10:48 2020 PUSH: Received control message: 'PUSH_REPLY,dhcp-option DNS 1.0.0.1,dhcp-option DNS 1.1.1.1,redirect-gateway def1 bypass-dhcp,tun-ipv6,route-ipv6 2000::/3,redirect-gateway ipv6,tun-ipv6,route-gateway 10.8.0.1,topology subnet,ping 10,ping-restart 120,ifconfig-ipv6 fd42:42:42:42::1000/112 fd42:42:42:42::1,ifconfig 10.8.0.2 255.255.255.0,peer-id 0,cipher AES-128-GCM'
Sun May 3 23:10:48 2020 OPTIONS IMPORT: timers and/or timeouts modified
Sun May 3 23:10:48 2020 OPTIONS IMPORT: --ifconfig/up options modified
Sun May 3 23:10:48 2020 OPTIONS IMPORT: route options modified
Sun May 3 23:10:48 2020 OPTIONS IMPORT: route-related options modified
Sun May 3 23:10:48 2020 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
Sun May 3 23:10:48 2020 OPTIONS IMPORT: peer-id set
Sun May 3 23:10:48 2020 OPTIONS IMPORT: adjusting link_mtu to 1624
Sun May 3 23:10:48 2020 OPTIONS IMPORT: data channel crypto options modified
Sun May 3 23:10:48 2020 Outgoing Data Channel: Cipher 'AES-128-GCM' initialized with 128 bit key
Sun May 3 23:10:48 2020 Incoming Data Channel: Cipher 'AES-128-GCM' initialized with 128 bit key
Sun May 3 23:10:48 2020 ROUTE_GATEWAY 192.168.1.1/255.255.255.0 IFACE=wlp2s0 HWADDR=98:5f:d3:d2:d3:49
Sun May 3 23:10:48 2020 GDG6: remote_host_ipv6=n/a
Sun May 3 23:10:48 2020 ROUTE6_GATEWAY fe80::8e97:eaff:fe02:c542 IFACE=wlp2s0
Sun May 3 23:10:48 2020 TUN/TAP device tun0 opened
Sun May 3 23:10:48 2020 TUN/TAP TX queue length set to 100
Sun May 3 23:10:48 2020 /sbin/ip link set dev tun0 up mtu 1500
Sun May 3 23:10:48 2020 /sbin/ip addr add dev tun0 10.8.0.2/24 broadcast 10.8.0.255
Sun May 3 23:10:48 2020 /sbin/ip -6 addr add fd42:42:42:42::1000/112 dev tun0
Sun May 3 23:10:48 2020 /sbin/ip route add 213.239.XXX.YYY/32 via 192.168.1.1
Sun May 3 23:10:48 2020 /sbin/ip route add 0.0.0.0/1 via 10.8.0.1
Sun May 3 23:10:48 2020 /sbin/ip route add 128.0.0.0/1 via 10.8.0.1
Sun May 3 23:10:48 2020 add_route_ipv6(2000::/3 -> fd42:42:42:42::1 metric -1) dev tun0
Sun May 3 23:10:48 2020 /sbin/ip -6 route add 2000::/3 dev tun0
Sun May 3 23:10:48 2020 add_route_ipv6(::/3 -> fd42:42:42:42::1 metric -1) dev tun0
Sun May 3 23:10:48 2020 /sbin/ip -6 route add ::/3 dev tun0
Sun May 3 23:10:48 2020 add_route_ipv6(2000::/4 -> fd42:42:42:42::1 metric -1) dev tun0
Sun May 3 23:10:48 2020 /sbin/ip -6 route add 2000::/4 dev tun0
Sun May 3 23:10:48 2020 add_route_ipv6(3000::/4 -> fd42:42:42:42::1 metric -1) dev tun0
Sun May 3 23:10:48 2020 /sbin/ip -6 route add 3000::/4 dev tun0
Sun May 3 23:10:48 2020 add_route_ipv6(fc00::/7 -> fd42:42:42:42::1 metric -1) dev tun0
Sun May 3 23:10:48 2020 /sbin/ip -6 route add fc00::/7 dev tun0
Sun May 3 23:10:48 2020 Initialization Sequence Completed
Comments
Well, with this little information we can't help much. The client conf looks ok as
Sun May 3 23:10:48 2020 /sbin/ip route add 0.0.0.0/1 via 10.8.0.1Sun May 3 23:10:48 2020 /sbin/ip route add 128.0.0.0/1 via 10.8.0.1shows that the correct route is routes are added. Have you got ufw / any other firewall installed? How about anything in /etc/iptables/rules.v4?
Maybe try the original script by @Nyr ?
https://github.com/Nyr/openvpn-install
The OS is Debian 10.3, I think the only firewall is iptables and here is the content of /etc/iptables/add-openvpn-rules.sh :
There is also /etc/iptables/rm-openvpn-rules.sh but no other script in this directory
I don't by what this was caused back then, however when I had a NAT VPS from @mikho I remember that angristans script for ovpn also did not work, however @Nyr script worked. Try @Nyr script
Ympker's VPN LTD Comparison, Uptime.is, Ympker's GitHub.
On the client side these are the routes before connecting to the VPN :
And here are the routes after connecting to the VPN :
I will try today but as the VPN connection does not look to be the issue it is really frustrating.
Try my installer, preferably in a clean server. It does both IP detection and the NAT in a better way, so that could be enough. It 100% works in LES.
It you are still having issues, we can troubleshoot from there, but the installer works on LES and you don't need to add any routing rule manually, just use the installer and specify the public IP when asked. Nothing more.
OpenVPN installer | WireGuard installer
Are you sure that the ip(6)tables rules are applied succesfully? On a fairly small number of my installations, the nat table requires the ip(6)tables-legacy instead of ip(6)tables and by the default PATH those executables couldn’t be found. Could you post the output of the ip(6)tables -L and ip(6)tables -t nat -L commands? Don’t forget to remove your IP addresses.
On a side note, I’d recommend @Nyr ’s script too. IMHO, having a systemd service for OpenVPN’s iptables rules make much more sense compared to a bash script, with the former at least you get some kind of observation chance about the application of the rules.
Just made a clean reinstall and use Nyr script, still no luck
Openvpn client log :
iptables -t nat -L
iptables -L
Edit :
ip6tables -L -t nat
ip6tables -L
I've seen issues with connectivity on VPS and IP(6)tables-legacy this before.
I would follow @berkay and check what's wrong with the settings.
https://clients.mrvm.net
How does the output of systemctl status openvpn-iptables.service look like?
sounds good :
The output of ip6tables (see above) makes me pretty confident that they are applied.
Well, they really look okay; I'm kind of clueless now. Did you check ip(6)tables-legacy tables too, as the output points out that they do have tables too?
All legacy tables look empy :
iptables-legacy -t nat -L
iptables-legacy -L
ip6tables-legacy -t nat -L
ip6tables-legacy -L
I would try adding the same rules with ip(6)tables-legacy too, you can find them on /etc/systemd/system/openvpn-iptables.service.
Not sure if this will help:
https://wiki.debian.org/nftables
reverting back to iptables instead of nftables.
https://clients.mrvm.net
If that fails, please try using a different distro (just as an environment test) such as CentOS.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
This is what I wanted to test, I will test this this evening.
OMG that was the problem since the beginning
it finally works. Thank you all folks for the valuable help !
There is probably an incompatibility between container iptables implementation and the host kernel version or something like that. Again, thanks a lot !
Debian breaks it with nftables
Centos or RHEL on the node doesn’t know how to handle it properly.
https://clients.mrvm.net
Thanks everyone for the information, I had no idea of the existence of this issue with OpenVZ.
I'm going to order an OVZ from @mikho to see exactly what distros and configurations are having this issue and add a check in the script to address it.
OpenVPN installer | WireGuard installer
It is simply that the 3.x VZ kernel does not support nf_tables, while I did experiment early on when debian 10 was released to load the modules it caused significant instability.
So you really need to add an if deb=10 then:
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
I see, will commit it tomorrow. I'm surprised that this issue has persisted for so long without a report reaching me.
OpenVPN installer | WireGuard installer
To be honest for all I know the nf_tables issues has been resolved in a later vz7/virtuozzo kernel, as I was a fairly early adopter of VZ7 in the LE* world and the fixes soon got fairly well documented so maybe a lot of people have just found that.
I don't fancy putting that theory to the test just for funzies though
Maybe it is just that hosts have later kernel versions and no nf_tables issues and enabled it in advance or no one is using Debian 10 in containers or ... and I suspect this to be a big part of it, the majority of hosts still use VZ6
Since from what I can tell most hosts are using my VZ7 templates anyway, the virtualizor templates were ones that I made i believe so i guess I could just make a new one with that already taken care of.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
Thanks for the detailed information, this has been addressed in the script.
OpenVPN installer | WireGuard installer
The only think worth mentioning just for a correctness stand point is that "nf_tables is not available in old OpenVZ kernels" should probably read "nf_tables is not available as standard in current OpenVZ kernels" as there are not really any old ones not in production VZ6 is 6 months+ EOL now. and current VZ7 follows the el7 kernel version pretty closely.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
I see, I mixed things up with you talking about the issues in previous kernels.
It's an important distinction as this means that it's possible that I'll need to keep addressing this with new distributions if they use iptables-nft by default.
OpenVPN installer | WireGuard installer
Great added value to the installation script @Nyr
Easy and understandble explanation @AnthonySmith
https://clients.mrvm.net