Help with HostHatch private network, can't see the other side
Struggling to get this running, and was hoping someone might be able to point me in the right direction.
I have enabled Private networking on old and new storage servers, and power cycled as instructed.
Old server is running Arch Linux, and uses netctl for networking.
New server currently Ubuntu. (will probably swap to Arch, want to get this working first).
Running ip -o link | grep 00:22 on both devices shows an ens4 device.
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\ link/ether 00:22:snip brd ff:ff:ff:ff:ff:ff\ altname enp0s4
On the Arch box, I have added an internal-network file with
Interface=ens4
Connection=ethernet
IP=static
Address=('192.168.10.3/24')
MACAddress=00:22:snip
// tried with and without the MACAddress line
On Ubuntu I have /etc/netplan/90-private.yaml per the docs https://docs.hosthatch.com/networking/#private-networking
network:
ethernets:
ens4:
addresses:
- 192.168.10.2/24
match:
macaddress: 00:22<skip>
version: 2
After enabling respective files, ip link returns showing ens4 is up.
Arch:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 00:16:snip brd ff:ff:ff:ff:ff:ff
altname enp0s3
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 00:22:snip brd ff:ff:ff:ff:ff:ff
altname enp0s4
Ubuntu:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 00:20:snip brd ff:ff:ff:ff:ff:ff
altname enp0s3
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 00:22:snip brd ff:ff:ff:ff:ff:ff
altname enp0s4
And the routing
Arch:
default via 31.snip dev ens3
31.snip/24 dev ens3 proto kernel scope link src 31.snip
192.168.10.0/24 dev ens4 proto kernel scope link src 192.168.10.3
Ubuntu
default via 31.snip dev ens3
31.snip/24 dev ens3 proto kernel scope link src 31.snip
192.168.10.0/24 dev ens4 proto kernel scope link src 192.168.10.2
But when I try and ping from one machine to the other I get destination unreachable.
What am I missing? Coz I'm baffled right now
Comments
Dumb question but since you don't say, they're at the same location, right?
tcpdump the interface on the destination side and see if you see the traffic coming in?
Good point, I forgot to specify they are both in Amsterdam. Although one is marked "Legacy".
Nothing appears when pinging from the Arch side to the Ubuntu side. The lines below are from me starting a ping on the Ubuntu side
Routing try ens4 not ens3
"I would have gotten away with it too, if it wasn't for that meddling Frankz and Mason!!"
I don't know Arch but it does not look like you have a route set for the private networking. So maybe this might help.
Ubuntu
ip route add 192.168.10.0/24 via 192.168.10.2 dev ens4
Arch:
ip route add 192.168.10.0/24 via 192.168.10.3 dev ens4
LES • About • Donate • Rules • Support
But the private addresses are on ens4 ... won't that just break his external network?
I have a feeling I'm missing something obvious but isn't that the highlighted lines?
When you say 'starting a ping on the Ubuntu side' you mean you have the tcpdump running on Ubuntu side and have Ubuntu try to ping Arch, right? So the Ubuntu machine is seeing it's own traffic ( guessing so since it's knows its name )? What happens if you tcpdump the Arch side and try the same?
Shouldn't it be
tcpdump -i ens4? I've never used a number like that...Although I am not any kind of routing expert, in my opinion, after the ip route commands I showed above are entered there should be additional route lines
192.168.10.0/24 via 192.168.10.2 dev ens4- on Ubuntu192.168.10.0/24 via 192.168.10.3 dev ens4- on ArchBecause right now I see everything going over the default route.
IIAC The lines you highlighted above show that the related IPs are UP and attached to dev ens4 the on the local machine, but not any routing.
LES • About • Donate • Rules • Support
They don't need any routing because the kernel knows what network it's on. You only need that if you've got access to something else via that network.
[root@cerebro ~]# ip addr show ens4
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc fq_codel state UP group default qlen 1000
link/ether 00:22:da:df:41:cf brd ff:ff:ff:ff:ff:ff
altname enp0s4
altname ens4
inet 10.174.2.24/8 brd 10.255.255.255 scope global noprefixroute eth1
valid_lft forever preferred_lft forever
[root@cerebro ~]# ip route
default via 193.29.62.1 dev eth0 proto static metric 100
10.0.0.0/8 dev eth1 proto kernel scope link src 10.174.2.24 metric 101
10.89.0.0/24 dev cni-podman1 proto kernel scope link src 10.89.0.1
192.168.0.0/16 dev nebula1 proto kernel scope link src 192.168.2.157 mtu 1400
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown
193.29.62.0/24 dev eth0 proto kernel scope link src metric 100
Sorry about the formatting. Stupid thing.
I'll just crawl back in my hole then.
LES • About • Donate • Rules • Support
No, no ... you should definitely check my work. Sorry if that sounded bad. I was as much as anything else saying it to make sure I wasn't wrong.
Edit: told you it was stupid
Try putting in the real Mac addresses
"I would have gotten away with it too, if it wasn't for that meddling Frankz and Mason!!"
Ah, good question.
My best guess is that they each accidentally got their own private network so they don't see each other. Thinking was the tcpdump's would show no traffic either way and since ARPs are broadcast no matter how buggered the OS setup was it should've seen it. But again, this isn't my area either.
I didn't take it badly, I just do not consider myself a routing expert and I work with RH based O/S so since this is Ubuntu and Arch I am out of my depth and though you knew what was best.
LES • About • Donate • Rules • Support
Normally it would but I don't know Hosthatch's setup on these. I was reading their article on it and it hit me but now I think he might not have the right MACs.
"I would have gotten away with it too, if it wasn't for that meddling Frankz and Mason!!"
phew Okay good. I feel better. Yeah, I'm a RHEL guy too. Was actually certified back on 6.
The one that has me curious is that he has a mix of old and new names on the Ubuntu box. I was under the impression it was all or nothing although I guess maybe netplan is mapping the name somehow.
Don't feel bad I work in Slackware. If it was Slack I would have his back. Yes I know it rhymes and is a pun as well. I don't utilize much private networking at least not this type of thing
"I would have gotten away with it too, if it wasn't for that meddling Frankz and Mason!!"
Started on Slackware back in .... 93'ish? Couldn't have been much later. Dropped out by 95 I think... Still have my 0.99 3.5 inch floppies though I'm sure they've got rampant bitrot by now ...
EDIT: rats, time for bed. Was just having fun too. >:(
Those were the days. No GUI command line interface by default. 8 hours of downloading to get the OS. Man we were so cool back then. Now we have become.... Ehhhhhh mainstream almost.
"I would have gotten away with it too, if it wasn't for that meddling Frankz and Mason!!"
OP seems gone from the conversation. @tarasis where are you? How are things going now ?
LES • About • Donate • Rules • Support
Probably tried to hard taking a dump and passed out, but seriously let us know what is going on! This tension is killing me.
"I would have gotten away with it too, if it wasn't for that meddling Frankz and Mason!!"
Did you check the firewall?
I have a same problem in Amsterdam and On OGF I saw the same problem which he said was on their side.
Apologies everyone for leaving the conversation. It was 2am and I crashed out, so you were kinda right @AuroraZero. Will have a read of all you have said and let you know status.
yes I have tried rebooting after making changes.
No firewall (yet) on Ubuntu, so it should receive pings without issue.
Yes firewall on Arch, but its open for ping from anywhere.
Nothing appears in the output from the Ubuntu box
but there is data from other sources
```[root@backup ~]# tcpdump -i ens4
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens4, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:06:40.156922 ARP, Request who-has 5. tell backup, length 28
14:06:40.182060 ARP, Request who-has one.one.one.one tell backup.tarasis.net, length 28
14:06:41.163887 ARP, Request who-has 5. tell backup, length 28
14:06:41.190542 ARP, Request who-has one.one.one.one tell backup.tarasis.net, length 28
14:06:42.177211 ARP, Request who-has 5. tell backup, length 28
14:06:42.203886 ARP, Request who-has one.one.one.one tell backup.tarasis.net, length 28
14:06:42.244691 ARP, Request who-has wine. tell backup, length 28
root@backup2tb:~# tcpdump -i ens4
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens4, link-type EN10MB (Ethernet), snapshot length 262144 bytes
13:14:29.756928 IP6 backup2tb > ip6-allrouters: ICMP6, router solicitation, length 16
13:17:38.343637 IP6 fe80::fc22:1ff:fec5:7af > ip6-allrouters: ICMP6, router solicitation, length 16
13:18:11.136796 IP6 backup2tb > ip6-allrouters: ICMP6, router solicitation, length 16
13:25:51.605649 IP6 backup2tb > ip6-allrouters: ICMP6, router solicitation, length 16
13:26:18.537889 IP6 fe80::fc22:1ff:fec5:7af > ip6-allrouters: ICMP6, router solicitation, length 16
[root@backup ~]# tcpdump -D
1.ens3 [Up, Running, Connected]
2.ens4 [Up, Running, Connected]
3.any (Pseudo-device that captures on all interfaces) [Up, Running]
4.lo [Up, Running, Loopback]
5.bluetooth-monitor (Bluetooth Linux Monitor) [Wireless]
6.nflog (Linux netfilter log (NFLOG) interface) [none]
7.nfqueue (Linux netfilter queue (NFQUEUE) interface) [none]
8.dbus-system (D-Bus system bus) [none]
9.dbus-session (D-Bus session bus) [none]
```
Seemingly
-iunderstands the 2 to match the output of-D.Slackware ... gosh that takes me back. I think I tried it and didn't get on and switched to RedHat around 1995 (we had solaris at uni, I wanted something unix esque as home so I could run some code at home in something "similar" to Uni). Then years later Suse, Ubuntu, Gentoo, Arch.
I was using the Mac reported by the
ip -o link | grep 00:22command. What I'm about to try is dropping the MAC address from the Ubuntu (Netplan) config.Interesting, what OS's are you going between?
Also thanks everyone for chipping in so far, greatly appreciated.
Thats my worry, and given their setup is basically a black box I will need to contact them I suspect. But want to try and rule out everything I can first.
So given what you said, its interesting that the Ubuntu side is seeing absolutely nothing on that interface but Arch does.
Going to eliminate part of the equation now and set up the Ubuntu box as Arch as well.
Yes, there seems to be an issue with private network in NL location (at least for BF sale storage VPSes). I'm not sure if anyone has an open ticket with HostHatch regarding this, so I've made one.
Yeah I hadn't yet, cheers @cyberpunk. Should I hold off also opening a ticket? Maybe just mention me in yours?
Just finished installing Arch on new box, and configured everything as per original Arch storage server and still no dice. The new storage server doesn't see anything at all on
ens4Where the old box starts seeing things within a few seconds.
I'm not really sure about internal HH policies and if extra tickets from affected clients actually help or slow down the process instead. Also, there is a good chance that the issue is already known to them, but very low on priority list for fixes, since it doesn't affect much provided service
@cyberpunk I'll wait then, I'd rather not poke more than I have to. Cheers.