Our server is an i9-9900K at Hetzner. Eight LXC containers are running various Linux distributions on Proxmox-VE 7.3. Everything seems to work great! . . . Except inside one Alpine Linux container. Somehow, IPv6 doesn't work from within this one container. Nevertheless, IPv6 ping to the container somehow seems to work from the Proxmox node. Amazingly, ping also works to the container even from the Wide Area Network (WAN).

How could such strangeness happen? Let's investigate!

A Closer Look At The Problem

The network configuration for all the containers was added individually by hand in the Proxmox GUI. Proxmox numbers LXC containers beginning with the number "100". The containers are IPv6 only. Each container was given a static IPv6 address ending with the container's Proxmox container number.

The container with the IPv6 network problem is number 102. An example of the working containers is number 106. Number 106 was chosen for comparison because both container 102 and container 106 were built from the same Alpine Linux LXC image. The other working containers are Ubuntu and Debian.

Alpine Linux seems less likely to be causing the IPv6 network failure because container 106, also built from the Alpine image, is functioning just fine. So let's look directly at the misbehaving container, number 102.

Container 102 can ping localhost.

root@Proxmox-VE ~ # lxc-attach -n 102
~ # ping6 -c 2 localhost
PING localhost (::1): 56 data bytes
64 bytes from ::1: seq=0 ttl=64 time=0.024 ms
64 bytes from ::1: seq=1 ttl=64 time=0.036 ms

--- localhost ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.024/0.030/0.036 ms
~ # 

Container 102 also responds to ping from external WAN.

[opc@instance-20220717-1620 ~]$ ping -c 2 2a01:4f8:121:24cc::102
PING 2a01:4f8:121:24cc::102(2a01:4f8:121:24cc::102) 56 data bytes
64 bytes from 2a01:4f8:121:24cc::102: icmp_seq=1 ttl=51 time=154 ms
64 bytes from 2a01:4f8:121:24cc::102: icmp_seq=2 ttl=51 time=154 ms

--- 2a01:4f8:121:24cc::102 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 153.966/154.042/154.118/0.076 ms
[opc@instance-20220717-1620 ~]$ 

However, from inside container 102, it can't ping its DNS servers, or the WAN.

root@Proxmox-VE ~ # lxc-attach -n 102
~ # cat /etc/resolv.conf
# --- BEGIN PVE ---
search metalvps.com
nameserver 2a01:4ff:ff00::add:2
nameserver 2001:470:20::2
# --- END PVE ---
~ # ping6 -c 2 2a01:4ff:ff00::add:2
PING 2a01:4ff:ff00::add:2 (2a01:4ff:ff00::add:2): 56 data bytes

--- 2a01:4ff:ff00::add:2 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
~ # ping6 -c 2 2001:470:20::2
PING 2001:470:20::2 (2001:470:20::2): 56 data bytes

--- 2001:470:20::2 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
~ # ping6 -c 2 metalvps.com
ping6: bad address 'metalvps.com'
~ # ping6 -c 2 ipv6.google.com
ping6: bad address 'ipv6.google.com'
~ # exit

By contrast, from inside working container 106, we can ping both the DNS servers and the WAN.

root@Proxmox-VE ~ # lxc-attach -n 106
~ # cat /etc/resolv.conf
# --- BEGIN PVE ---
search metalvps.com
nameserver 2a01:4ff:ff00::add:2
nameserver 2001:470:20::2
# --- END PVE ---
~ # ping6 -c 2 2a01:4ff:ff00::add:2
PING 2a01:4ff:ff00::add:2 (2a01:4ff:ff00::add:2): 56 data bytes
64 bytes from 2a01:4ff:ff00::add:2: seq=0 ttl=61 time=0.446 ms
64 bytes from 2a01:4ff:ff00::add:2: seq=1 ttl=61 time=0.274 ms

--- 2a01:4ff:ff00::add:2 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.274/0.360/0.446 ms
~ # ping6 -c 2 2001:470:20::2
PING 2001:470:20::2 (2001:470:20::2): 56 data bytes
64 bytes from 2001:470:20::2: seq=0 ttl=57 time=18.516 ms
64 bytes from 2001:470:20::2: seq=1 ttl=57 time=18.552 ms

--- 2001:470:20::2 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 18.516/18.534/18.552 ms
~ # ping6 -c 2 metalvps.com
PING metalvps.com (2603:c020:3:a9a9::250): 56 data bytes
64 bytes from 2603:c020:3:a9a9::250: seq=0 ttl=49 time=151.895 ms
64 bytes from 2603:c020:3:a9a9::250: seq=1 ttl=49 time=151.856 ms

--- metalvps.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 151.856/151.875/151.895 ms
~ # ping6 -c 2 ipv6.google.com
PING ipv6.google.com (2a00:1450:4001:830::200e): 56 data bytes
64 bytes from 2a00:1450:4001:830::200e: seq=0 ttl=118 time=5.477 ms
64 bytes from 2a00:1450:4001:830::200e: seq=1 ttl=118 time=5.611 ms

--- ipv6.google.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 5.477/5.544/5.611 ms
~ # 

Could The Firewall Cause This Problem?

The problem with container 102 probably is not a firewall drop egress issue because there aren't any egress rules in the container's zone and the default egress policy is ACCEPT. Also, the container's internal IPv6 egress problem still happens if container 102's firewall zone is set entirely off.

The ip Command

The ip command from the iproute2 suite provides much information about Linux networking. As the ip(8) man page explains, we can use ip in the following syntax: ip [OPTIONS] OBJECT { COMMAND | help }. In other words, to use ip we need to specify three items: an OPTION, an OBJECT, and a COMMAND.

For IPv4, the usual OPTION is nothing because ip defaults to IPv4. But, if we wish to specify the IPv4 OPTION explicitly, the option could be specified as "-family inet". The IPv4 OPTION can be abbreviated to "ip -f inet" or just "ip -4".

For IPv6, we use the OPTION "ip -family inet6". The IPv6 OPTION can be abbreviated to "ip -f inet6" or just "ip -6".

The OBJECT can be, for example, "link", "address", or "route". A "link" is a hardware or virtualized interface connecting our machine to a network. The "address" is the numerical IP address or addresses we assigned to our link. The "route" is the IP address where the kernel should send network packets initiated by programs running on our machine. "link", "address", and "route" can be abbreviated as "l", "a", and "r", respectively. Longer abbreviations also work. For example, "addr" commonly is used as an abbreviation for "address".

Frequently, when something happens involving networking, it can be very helpful to check the output of the ip command. Let's look with the ip command and see what we can find!

The Error Message

Checking the output of the ip -family inet6 address show command inside container 102 reveals "tentative dadfailed" as follows.

root@Proxmox-VE ~ # lxc-attach -n 102
~ # ip -f inet6 addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0@if45: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 state UP qlen 1000
    inet6 2a01:4f8:121:24cc::102/64 scope global tentative dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::d8dd:65ff:fe45:70f2/64 scope link 
       valid_lft forever preferred_lft forever
~ # exit
root@Proxmox-VE ~ # 

Output of the same command from inside container 106 where networking works fine does not include "tentative dadfailed."

root@Proxmox-VE ~ # lxc-attach -n 106
~ # ip -f inet6 addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0@if28: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 state UP qlen 1000
    inet6 2a01:4f8:121:24cc::106/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::64ce:78ff:fe85:15fd/64 scope link 
       valid_lft forever preferred_lft forever
~ # 

Dadfailed

Some quick googling found a page called IPv6 'dadfailed' Problem. The problem there involved reassigning an address previously assigned to another machine which failed without the address being deleted. However, down toward the bottom of the page, we find key information: the "dad" in "dadfailed" is "Duplicate Address Detection."

No Mistake With Container 102's Creation

But, even after reading about Duplicate Address Detection, I still did not immediately find the solution. I was puzzled about how, like the dadfailed page that Google showed me, some machine possibly could be down but with its address still present on my clean, new server with all its containers working fine except for this single itty bitty networking issue inside one container. Nothing was down. All the containers had their own addresses, and all the containers successfully started and continued running.

I kept looking and looking for some mistake I somehow might have made while setting up container 102. I checked and recheckd the screenshots that recorded container 102's creation. But everything looked okay!

Asking For Help

Next I talked with a well known hosting provider about what was happening. I also wrote up a LES post asking for help with the puzzling issue. It was starting to get late by the time I finished drafting the LES post. I decided to sleep. I deferred posting to LES Help until the next morning.

When I woke up, however, the idea was in my mind that possibly I mistakenly also had assigned the 102 address to another container. Such a mistake didn't seem likely, though, because there were only 8 containers, and the container numbers, 100, 101, 102, 103, 104, 105, 106, and 107 seemed understandable. I decided to check all the other containers besides 102 to make sure no additional container inadvertently and duplicatively also had been assigned the 102 address.

Checking All The Containers

• Container 100

First I checked container 100. Container 100 was the test container that had been checked by @yoursunny and found to be "very strong." Container 100 seemed to have been assigned the correct IP address ending in 100.

• Container 101

Uh! Oh! Here it is! Container 101 got the 102 address!!

How Did The Error Happen?

Container 101 was the second container that was created. Obviously it should have an IP address ending with 2, right? Well, not when the first address ended with 0.

What About The Node And WAN Responses To Ping?

How could container 102 respond to a ping from external WAN while seeming unable to initiate a ping to an external WAN destination? Container 101 already had been created with the 102 address. So, the ping responses came from container 101.

How Do We Fix container 102?

All that's needed to fix container 102 is to change its address to any that is not already in use. I tried the 101 address, and that address immediately worked fine.

More Information About Dadfailed

The ip command has a manual page which doesn't mention dadfailed. However, the ip man page refers to the ip-address man page, which does mention dadfailed as an addition to the ip address show command.

   ip address show - look at protocol addresses

        [ . . . ]

       dadfailed
              (IPv6 only) only list addresses which have failed duplicate ad‐
              dress detection.

       -dadfailed
              (IPv6 only) only list addresses which have not failed duplicate
              address detection.

For example, inside the Linux container on my Chromebook:

chronos@penguin:~/log$ ip -6 address show dadfailed
chronos@penguin:~/log$ ip -6 address show -dadfailed
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
5: eth0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::216:3eff:fe43:22fb/64 scope link 
       valid_lft forever preferred_lft forever
chronos@penguin:~/log$ 

Dadfailed In The RFCs

Duplicate Address Detection failure is discussed in section 5.4.5 of RFC 4862 which says:

5.4.5. When Duplicate Address Detection Fails

A tentative address that is determined to be a duplicate as described above MUST NOT be assigned to an interface, and the node SHOULD log a system management error.

Further discussion of Duplicate Address Detection occurs in Appendix A of RFC 4862 and in RFC 7527 Enhanced Duplicate Address Detection.

Dadfailed In The Logs

Indeed, the node did log a system management failure just as RFC 4862 said it should:

root@Proxmox-VE /var/log # zcat messages.2.gz | grep "duplicate address"
Jan 10 03:01:27 Proxmox-VE kernel: [97048.519242] IPv6: eth0: IPv6 duplicate address 2a01:4f8:121:24cc::102 used by 92:17:7a:22:c1:e2 detected!
[ . . . ]
Jan 13 06:44:36 Proxmox-VE kernel: [  141.636572] IPv6: eth0: IPv6 duplicate address 2a01:4f8:121:24cc::102 used by 92:17:7a:22:c1:e2 detected!
root@Proxmox-VE /var/log # 

Source Code

Recently I started to look at C source code. I don't know hardly anything about C, so looking at source code always is an adventure.

Above, we previously saw the ip command print the following "dadfailed" output:

~ # ip -f inet6 addr show
  [ . . . ]
2: eth0@if45: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 state UP qlen 1000
    inet6 2a01:4f8:121:24cc::102/64 scope global tentative dadfailed 
       valid_lft forever preferred_lft forever
  [ . . . ]

Can we find the source code which printed the dadfailed error? Finding the printing code sounds simple enough. Probably the printing code is nowhere near as complicated as the code which checked for and found the dadfailed error, right?

We saw the dadfailed error notification inside an Alpine container. Possibly that error notification came from iproute2 code running inside Busybox inside the Alpine container. Nevertheless, let's start by imagining how we might get the iproute2 source code that Proxmox itself uses. I'm unsure the method here gets the sources correctly, but we can try. First come the Proxmox sources, which seem to include the Debian sources, and then we get what might be the upstream sources.

root@Proxmox-VE ~ # git clone git://git.proxmox.com/git/iproute2.git
Cloning into 'iproute2'...
remote: Enumerating objects: 187, done.
remote: Total 187 (delta 0), reused 0 (delta 0), pack-reused 187
Receiving objects: 100% (187/187), 11.25 MiB | 88.63 MiB/s, done.
Resolving deltas: 100% (83/83), done.
root@Proxmox-VE ~ # 
cd iproute2/
root@Proxmox-VE ~/iproute2 # ls
debian  iproute2  Makefile  README
root@Proxmox-VE ~/iproute2 # cat README 
We compile our own package to make sure we always have the
latest version and bug fixes.
root@Proxmox-VE ~/iproute2 # ls debian/
changelog  copyright          iproute2-doc.examples  iproute2.links     README.Debian  source
compat     doc                iproute2-doc.install   iproute2.manpages  README.source
control    iproute2-doc.docs  iproute2.install       patches            rules
root@Proxmox-VE ~/iproute2 # ls iproute2/ #The iproute2/iproute2 directory is empty.
root@Proxmox-VE ~/iproute2 # git clone git://git.kernel.org/pub/scm/network/iproute2/iproute2.git
Cloning into 'iproute2'...
remote: Enumerating objects: 313, done.
remote: Counting objects: 100% (313/313), done.
remote: Compressing objects: 100% (289/289), done.
remote: Total 33973 (delta 60), reused 37 (delta 20), pack-reused 33660
Receiving objects: 100% (33973/33973), 8.34 MiB | 42.06 MiB/s, done.
Resolving deltas: 100% (25490/25490), done.
root@Proxmox-VE ~/iproute2 # 

Let's look inside the iproute2/iproute2 directory and then inside the iproute2/iproute2/ip directory.

root@Proxmox-VE ~/iproute2 # cd iproute2/
root@Proxmox-VE ~/iproute2/iproute2 # ls
bash-completion  COPYING  doc       genl     lib       misc   README        tc         vdpa
bridge           dcb      etc       include  Makefile  netem  README.devel  testsuite
configure        devlink  examples  ip       man       rdma   schema        tipc
root@Proxmox-VE ~/iproute2/iproute2 # cd ip
root@Proxmox-VE ~/iproute2/iproute2/ip # ls
ila_common.h           iplink.c            iplink_vlan.c    ipprefix.c          link_vti.c
ip6tunnel.c            iplink_can.c        iplink_vrf.c     iproute.c           link_xfrm.c
ipaddress.c            iplink_dsa.c        iplink_vxcan.c   iproute_lwtunnel.c  Makefile
ipaddrlabel.c          iplink_dummy.c      iplink_vxlan.c   iprule.c            nh_common.h
ip.c                   iplink_geneve.c     iplink_wwan.c    ipseg6.c            routel
ip_common.h            iplink_gtp.c        iplink_xdp.c     ipstats.c           rtm_map.c
ipfou.c                iplink_hsr.c        iplink_xstats.c  iptoken.c           rtmon.c
ipila.c                iplink_ifb.c        ipmacsec.c       iptunnel.c          static-syms.c
ipioam6.c              iplink_ipoib.c      ipmaddr.c        iptuntap.c          tcp_metrics.c
ipl2tp.c               iplink_ipvlan.c     ipmonitor.c      ipvrf.c             tunnel.c
iplink_amt.c           iplink_macvlan.c    ipmptcp.c        ipxfrm.c            tunnel.h
iplink_bareudp.c       iplink_netdevsim.c  ipmroute.c       link_gre6.c         xfrm.h
iplink_batadv.c        iplink_nlmon.c      ipneigh.c        link_gre.c          xfrm_monitor.c
iplink_bond.c          iplink_rmnet.c      ipnetconf.c      link_ip6tnl.c       xfrm_policy.c
iplink_bond_slave.c    iplink_team.c       ipnetns.c        link_iptnl.c        xfrm_state.c
iplink_bridge.c        iplink_vcan.c       ipnexthop.c      link_veth.c
iplink_bridge_slave.c  iplink_virt_wifi.c  ipntable.c       link_vti6.c
root@Proxmox-VE ~/iproute2/iproute2/ip # 

It was the address OBJECT which drew our interest, so let's look at ipaddress.c.

root@Proxmox-VE ~/iproute2/iproute2/ip # cat -n ipaddress.c | grep dadfailed
    68                  "           [-]tentative | [-]deprecated | [-]dadfailed | temporary |\n"
  1411          { .name = "dadfailed",          .mask = IFA_F_DADFAILED,        .readonly = true,      .v6only = true},
root@Proxmox-VE ~/iproute2/iproute2/ip # 

Line 68 seems to be part of the error message which is printed when ip doesn't understand the arguments given to it on the command line.

If we back up from line 1411 to line 1400, we can see the comment reproduced here, just below. Also, a little further down, at lines 1451 and 1452, there's code involving both "print" and "name." I'm guessing that maybe lines 1451 and 1452 might be responsible for printing "tentative" and "dadfailed."

  1400  /* Mapping from argument to address flag mask and attributes */
  1401  static const struct ifa_flag_data_t {
  [ . . . ]
  1411          { .name = "dadfailed",          .mask = IFA_F_DADFAILED,        .readonly = tru
e,       .v6only = true},
  [ . . . ]
  1414          { .name = "tentative",          .mask = IFA_F_TENTATIVE,        .readonly = tru
e,       .v6only = true},
  [ . . . ]
  1451                                  print_string(PRINT_FP, NULL,
  1452                                               "%s ", flag_data->name);

Finally, glancing at the Busybox ip command, the relevant source code seems to be ip.c. The usage message for ip address show is on lines 139 to 149.

139  //usage:#define ipaddr_trivial_usage
140  //usage:       "add|del IFADDR dev IFACE | show|flush [dev IFACE] [to PREFIX]"
141  //usage:#define ipaddr_full_usage "\n\n"
142  //usage:       "ipaddr add|change|replace|delete dev IFACE [CONFFLAG-LIST] IFADDR\n"
143  //usage:       "   IFADDR := PREFIX | ADDR peer PREFIX [broadcast ADDR|+|-]\n"
144  //usage:       "       [anycast ADDR] [label STRING] [scope SCOPE]\n"
145  //usage:       "   PREFIX := ADDR[/MASK]\n"
146  //usage:       "   SCOPE := [host|link|global|NUMBER]\n"
147  //usage:       "   CONFFLAG-LIST := [CONFFLAG-LIST] CONFFLAG\n"
148  //usage:       "   CONFFLAG := [noprefixroute]\n"
149  //usage:       "ipaddr show|flush [dev IFACE] [scope SCOPE] [to PREFIX] [label PATTERN]"

There is more on lines 341 to 347.

341  #if ENABLE_IPADDR
342  int ipaddr_main(int argc, char **argv) MAIN_EXTERNALLY_VISIBLE;
343  int ipaddr_main(int argc UNUSED_PARAM, char **argv)
344  {
345     return ip_do(do_ipaddr, argv);
346  }
347  #endif

ip_do is defined on lines 334 to 338.

334  static int ip_do(ip_func_ptr_t ip_func, char **argv)
335  {
336     argv = ip_parse_common_args(argv + 1);
337     return ip_func(argv);
338  }

Hopefully I soon will be able to understand more about how the iproute2 and Busybox code works to print the dadfailed error. Probably someone here at LES will be kind enough to provide some hints! Thanks very much! It is a lot of fun to learn a little about dadfailed!

What I want to reach is to make IPv6 work in my home network and I tried several ways to make the tunnel working, but both do not work.

First approach: using NAT IP (192.168.88.1)
[admin@MikroTik] > /interface 6to4 add comment="Route48.org Tunnel Broker (DE)" disabled=no local-address=192.168.88.1 mtu=1280 name=sit1 remote-address=194.50.X.X [admin@MikroTik] > /ipv6 route add comment="" disabled=no distance=1 dst-address=::/0 gateway=2a06:a003:XXXX::1 scope=30 target-scope=10 [admin@MikroTik] > /ipv6 address add address=2a06:a003:XXXX::2/48 advertise=no disabled=no eui-64=no interface=sit1

Second approach: using carrier router IP (192.168.1.6):
[admin@MikroTik] > /interface 6to4 add comment="Route48.org Tunnel Broker (DE)" disabled=no local-address=192.168.1.6 mtu=1280 name=sit1 remote-address=194.50.X.X [admin@MikroTik] > /ipv6 route add comment="" disabled=no distance=1 dst-address=::/0 gateway=2a06:a003:XXXX::1 scope=30 target-scope=10 [admin@MikroTik] > /ipv6 address add address=2a06:a003:XXXX::2/48 advertise=no disabled=no eui-64=no interface=sit1

I also tried to forward all ports to Mikrotik and use my public IPv4, but it didn't work neither...

After any try, I tried to ping Cloudflare IPv6 DNS, but there are timeouts...
[admin@MikroTik] > ping 2606:4700:4700::1111 SEQ HOST SIZE TTL TIME STATUS 0 2606:4700:4700::1111 timeout 1 2606:4700:4700::1111 timeout 2 2606:4700:4700::1111 timeout 3 2001:470:71:74b:: 104 64 95ms158us address unreachable

What am I doing wrong in my config? Tunnel does not work neither with Route48 nor HE.

/23 Usage Type: Residential IP / (ISP) Fixed Line ISP
/23 Usage Type: (COM) Commercial

It will be for private services use where datacenter IPs (DCH) are not accepted.
IPs must be valid and updated in IP2Location.

It will not be for prohibited traffic use like spam, torrents, hosting, hidden proxy etc.

Payment methods: PayPal or credit card.

Interested, please leave a quote via PM or contact email.

It all started the day I dreamed of trying lower level dedicated server network and virtual machine configuration by myself instead of having Proxmox do the configuration for me. So, recently, I moved my MetalVPS server from Proxmox to Debian sid.

As part of the move from Proxmox to Debian sid, I got my clueless self into big trouble when I imagined setting up link layer networking inside the server. I imagined that, with link layer networking, virtual machines could use all sixteen of the server's expensive IPv4 addresses. Routing the extra subnet inside the server would have cost three IP addresses from the subnet, leaving only thirteen available for use by virtual machines. I wanted to use all 16 IPs.

Entire days passed by while I kept trying different network configurations. Eventually it seemed like I must have tried every possible configuration except the one configuration that would have worked! I kept reading and rereading the Hetzner tutorials plus several others that I found. Even posting here at LES and getting great responses wasn't enough to push me to victory.

Ultimately there seemed no choice other than to get help. But, from whom? I asked friends for referrals. I looked on Google. I checked Upwork and Fiverr. Most of the search results were about Microsoft, but I wanted Linux.

Eventually, however, I struck pure gold when I searched for "linux server administration -microsoft". There was this guy, Ratko, who seemed listed everywhere on every freelance site. And all the Ratko pages mentioned "networking." It seemed like Ratko was an exact fit for my problem!

But, which freelancing website to use? I've never used any of them, and so, as usual, I was clueless. I wondered if I could make direct contact with Ratko. It took awhile to figure out his last name. And then I found an email address! So I sent off an email.

Wow! Ratko replied almost immediately! Now, after working with Ratko, I know he always replies almost immediately. I don't know how he replies so fast! Well, "immediately" might be overstating the case. Maybe ten or twenty minutes. But that's amazing!

Ratko seemed to think he could fix my server's network configuration so that my plan to use all sixteen IP addresses in the subnet would work.

I made an account on the server for Ratko. Haha! I cluelessly forgot to install the ssh public key that Ratko had sent me at my request. Haha!

After he finally could log in, Ratko suggested changes in my attempted network configuration plus a few changes in Hetzner's or Debian's kernel network configuration and their qemu defaults, plus also a change in my qemu VM launch script.

Haha, well, nothing worked, but that was, as Ratko realized, because I had not left even one VM running.

Next it was time to look inside a running VM. I set one up. Ratko courageously and tirelessly fought through all the trouble of setting up an ssh tunnel! He then dove inside the test VM through qemu's VNC console interface!

Well, you will not be surprised that Ratko made a couple of changes to my VM configuration. And then. . . .

Suddenly, everything worked! It was like magic! Ratko comes in, looks carefully but quickly at what I had been trying for weeks. Then he makes a tweak here, there, everywhere. Suddenly, everything just works!

Now a few days and a few reboots, retests, and IP changes later, Ratko's entire configuration seems stable and indeed does appear to work for all the IPs in the subnet. Plus, no complaints from Hetzner!

Next came time for financial reckoning, Ratko asked me to send him an amount I thought was fair. While thinking about what number might be fair, I happened to notice a three digit number in one of the octets of the IP address of our test VM. Although it was an odd number, it nevertheless jumped out as perfect! I immediately sent Ratko that exact number of dollars.

Ratko told me the amount was generous. I won't say for sure that the funds I sent to Ratko were my best spent money ever, but I do say for sure that they might have been.

Working with Ratko is pure joy! Everything remains calm, but there is speed, focus, intensity, nuance, breadth, depth, vast context, wide experience, patience, courtesy, and great generosity! Plus perfect results!

Here is a link to Ratko's LinkedIn.

Highest thanks to Ratko for a super great job! Five stars! ⭐⭐⭐⭐⭐

Afterword

My biggest problem writing this review is imagining people won't believe what I am saying about Ratko because my review is too good to be true. Well, every language's literature has its classic hero saga genre. So now maybe we have a Beowulf for Linux network and system administration freelancers and their worthy clients.

I am delighted to speak privately with anyone considering hiring Ratko. If you contact me I promise additional superlatives! 🤩

I sent Ratko a few questions in case he wanted to answer them for you guys. I did not include a draft of this review. Ratko's answers are below.

For those interested, a few more tech details are posted here and elsewhere throughout that same thread.

Friendly greetings from New York City and Sonora, MX! 🗽🇺🇸🇲🇽🏜️

Interview with Ratko @PenguinGenius

Hi Tom,

Hey Ratko!
Hope you are doing great!

So far so good

How did you get started with Linux?

25+ years ago With Slackware 1.0 distro.

How did you get started with networking?

About same time, 1996. when I started working in Data Center, and I was
at Junior Network Engineer position first.

How did you get started freelancing?

Couple years after I got my regular job at University, as government
doesn't give high salaries
Friend of mine asked me to start as QA Engineer for his friends company,
which build some shareware applications for Windows.

How do you and your clients find each other?

Freelancer sites at first, LinkedIn, and finally - satisfied clients
recommendation

What has been your most complex and challenging freelance job?

Can't remember, I'm doing this for so many years...

How could your clients make your life easier?

With clear and complete info about problem, access, etc. And timely
response for my additional queries.

What's your schedule on days, nights, and weekends?

8am-3pm - in office at University - sometimes doing some fast freelance
tasks in office when I'm bored.
7pm-10pm - freelance work for clients (or more if needed)
Night - only if world catastrophic failure is in progress
Weekends - only super urgent requests (which of course have super-duper
high hourly rate), or some maintenance tasks I have scheduled over
weekend days.

If you also have a regular job, how do you manage conflicts?

I do have regular job. No conflicts - office work is primary. Then
clients' requests scheduled per urgency.
But in office first rule of system administration helps: "Best sysadmin
is one who is not doing anything!"
Meaning - he did all great, and nothing to do more (until any failure).

How can you respond to clients so quickly?

I know my value, and what I can do
And I'd like any my client to feel special and to know I take care for
him.

What advice would you give a new freelancer?

Don't Learn to Hack... Hack to Learn!
Even something seems hard - try to apply RTFM rule (Read The Fuckin'
Manual) before start to ask around. Try things (and know how to revert).
And don't feel insecure in yourself - otherwise, this is not for you.

Hetzner seems to have an interesting policy that outgoing packets from virtual machines ("VMs") using additional subnet IPs on dedicated servers must come from the known-to-Hetzner hardware MAC address of the server.

Thus, Hetzner wants VMs using subnet IPv4 addresses to use the server node's main IPv4 as the VM's gateway. If I understand correctly, the usual way to accomplish using the node's main IPv4 as a VM gateway means that the VMs cannot use all of the IP addresses in the node's IPv4 subnet.

However, could using layer 2 networking for IPv4 on the server node allow use of all of the additional IPv4 subnet addresses by VMs while still meeting Hetzner's requirements?

This enticing possibility is suggested at https://www.sysorchestra.com/hetzner-root-server-with-kvm-ipv4-and-ipv6-networking/ .

The subnet here is a /28 with 16 IP addresses. Would it be completely crazy to image that an /etc/network/interfaces configuration somewhat like the following might work to utilize all 16 of the IPv4 subnet IPs?

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback
iface lo inet6 loopback

auto enp7s0
iface enp7s0 inet static
  address 198.18.1.0  
  netmask 255.255.255.255
  gateway 198.18.0.1
  pointopoint 198.18.0.1
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up echo 0 > /proc/sys/net/ipv4/conf/enp7s0/send_redirects

iface enp7s0 inet6 static
  address 2001:0002::2
  netmask 128
  gateway fe80::1
    post-up echo 1 > /proc/sys/net/ipv6/conf/all/forwarding

auto vmbr0
iface vmbr0 inet static
  address 198.18.1.0 
  netmask 255.255.255.255
  bridge_ports none
  bridge_stp off
  bridge_fd 0
  bridge_maxwait 0
    post-up ip route add 198.18.10.0/32 dev vmbr0
    pre-down ip route del 198.18.10.0/32 dev vmbr0
    post-up ip route add 198.18.10.1/32 dev vmbr0
    pre-down ip route del 198.18.10.1/32 dev vmbr0
      [ . . . ]
    post-up ip route add 198.18.10.15/32 dev vmbr0
    pre-down ip route del 198.18.10.15/32 dev vmbr0

iface vmbr0 inet6 static
  address 2001:0002::2
  netmask 64

When the server is booted using the above /etc/network/interfaces configuration, and with no VMs running, I seem to see something like the following. Is this as should be expected?

not-oles@server:~$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether DE:AD:BE:EF:DE:AD brd ff:ff:ff:ff:ff:ff
3: vmbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether  DE:AD:BE:EF:BE:AD brd ff:ff:ff:ff:ff:ff
not-oles@server:~$ ip route show
default via 198.18.0.1 dev enp7s0 onlink 
198.18.10.0 dev vmbr0 scope link linkdown 
198.18.10.1 dev vmbr0 scope link linkdown 
  [ . . . ]
198.18.10.15 dev vmbr0 scope link linkdown
not-oles@server:~$ ip -6 link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether DE:AD:BE:EF:DE:AD brd ff:ff:ff:ff:ff:ff
3: vmbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether DE:AD:BE:EF:BE:AD brd ff:ff:ff:ff:ff:ff
not-oles@server:~$ ip -6 route show
::1 dev lo proto kernel metric 256 pref medium
2001:0002::2 dev enp7s0 proto kernel metric 256 pref medium
2001:0002::/64 dev vmbr0 proto kernel metric 256 linkdown pref medium
fe80::/64 dev enp7s0 proto kernel metric 256 pref medium
default via fe80::1 dev enp7s0 metric 1024 onlink pref medium
not-oles@server:~$ 

Thanks very much for reading and for any help! 💛 Best wishes from Mexico's Sonoran Desert! 🚵

Here is a short introduction of the three, in case you don't know them yet:

• ping.pe - some 30 locations, mostly NA and China, also some EU and a bit of APAC
• dnstools.ws - some 30 locations, much more diverse than ping.pe (the majority excl. US and AU have 1 location per country = there's more countries in total), areas included: NA, EU, APAC, Africa (ZA)
• mtr.sh - a shitload (200+?) of locations worldwide, including LATAM (MX, CO, BR, PE, CL, AR)

I. Before We Start

We need to obtain our basic network configuration from our provider. Or, if we are running our own host node, we need to assign basic network configuration to ourselves. Our basic network configuration might look something like this:

For IPv6, one might expect something like:

But occasionally, IPv6 could be something like:

Notes:

• The /28 in the IPv4 address and the longer netmask are different ways of providing the same information about the size of the local, directly connected network. It suffices for us to have this information in one format or the other. We don't need both formats because the information is the same. Also, the broadcast IP might not be provided, since it isn't strictly necessary.
• For the second format of the IPv6 address, what happened to the /64? 😱 The /128 in the second form of the IPv6 address might seem clueless to IPv6 fans expecting a /64. Also, the second format of the IPv6 address includes a gateway6 address. The gateway6 address might seem strange to some IPv6 fans, but we need the gateway6 for our minimal, static configuration. More on all this below.

II. Introduction

In the previous post of this series we finished using the Proxmox web GUI to install our new Debian KVM VPS via the Debian netinst installer iso image. The final step in Part II was removing the netinst install iso image from the emulated cdrom and then reooting our new VM, which came up from its own internal filesystem:

In today's post, we continue from this exact place where we left Part II -- connected to our newly installed and newly rebooted KVM via the Proxmox web GUI. In this post, we will accomplish the networking configuration which was skipped in Part II because the Debian netinst iso doesn't automatically configure out of band IP addresses.

There are three network configuration and related tasks we will accomplish today:

• First, we go "inside" our VM through the Proxmox web GUI's emulated "physical" console connection and set up networking. In Debian, networking setup requires that we adjust the file /etc/network/interfaces to tell our VM its network address and the address of its gateway to the internet.
• Second, we edit the file /etc/resolv.conf to tell our VM the numerical addresses of Domain Name System ("DNS") servers it can use to translate human readable Uniform Resource Identifiers (URI) into numerical Internet Protocaol ("IP") addresses.
• Third, we set up /etc/apt/sources.list to tell our system's Aptitude software package manager ("APT") where to get software updates and the additional software packages we will want to install.

Section III, Quick Setup, runs quickly through all three of today's tasks in "recipe style."

Section IV offers additional context on our setup environment.

Sections V, VI, and VII provide additional details on today's three setup tasks.

Section VIII discusses security.

Section IX discusses backup.

When we finish the Quick Setup, our new Debian KVM VPS should be connected to the internet, DNS should work, and we should be able to use the Debian package system to add whatever additional software we want.

When we finish all of today's post, we should have reasonable context within which to understand our Debian VM's networking setup.

III. Quick Setup

Logged into our VM through the Proxmox web GUI, we run the command ip link show. This command will give us the name of our network interface, probably something like "ens18."

As root or with sudo, we edit the text of the file /etc/network/interfaces so that it contains the minimum necessary information:

auto ens18
iface ens18 inet static
  address IPv4_ADDRESS/CIDR
  gateway GATEWAY_ADDRESS

iface ens18 inet6 static
  address IPv6_ADDRESS/CIDR
  gateway GATEWAY6_ADDRESS

Using our example network configuration, our minimal /etc/network/interfaces looks like this:

auto ens18
iface ens18 inet static
  address 172.16.165.97/28
  gateway 172.16.164.1

iface ens18 inet6 static
  address fe80:xxxx:xxxx:xxxx::97/128
  gateway fe80:xxxx:xxxx:xxxx::3

Second, we edit the /etc/resolv.conf file so that it looks like this:

nameserver 1.1.1.1
nameserver 8.8.8.8
nameserver 2606:4700:4700::1111
nameserver 2001:4860:4860::8888

Third, we edit /etc/apt/sources.list so that it looks like this:

deb http://deb.debian.org/debian buster main contrib non-free

deb http://deb.debian.org/debian-security/ buster/updates main contrib non-free

deb http://deb.debian.org/debian buster-updates main contrib non-free

Finally, we restart networking so that our new configuration takes effect:

systemctl restart networking

At this point, we should have both IPv4 and IPv6 connectivity, and DNS and APT both should work.

IV. More Context

• Virtualized Console Connection

The Proxmox web GUI virtualizes a wired console connection. In other words, our web browser does connect over the internet to our Proxmox server, but, the view from inside our new KVM is the same as though a wired connection was attached. Our new KVM thinks it's talking over a wired connection to a physical console. From inside our new KVM, there is, as yet, no network connection.

By default, the Proxmox web GUI works via VNC. In the Proxmox wiki on serial terminal Proxmox warns that VNC might

not have the features you need (i.e. easy copy/paste between other terminals)

or it might be

impossible to capture all [kernel messages, standard output, or error] messages on [the] VNC screen.

Yep, copy / paste commands do not seem to work in the Proxmox KVM virtual console.

Also, if you enjoy using the vi editor, you might find what looks like a "Send-Esc" button among the set of choices within the set exposed by the top button on the console VNC control bar. Use of the real keyboard Escape key results in exiting full screen. However, a second real Esc seems to produce the expected mode change, despite that maybe we no longer can see too well without returning to full screen.

• No DHCP, No SLAAC

These days most network setups use Dynamic Host Configuration Protocol (DHCP) to autoconfigure IPv4 networking. The machine on which networking is to be configured asks for and receives from a DHCP server all the needed information for the networking setup.

It is possible to configure DHCP so that it always returns the same IP address to each VM, but, since our entire Proxmox network is static, it may be simpler to set up networking manually--the traditional way for servers.

Stateless Address Autoconfiguration ("SLAAC") provides automatic configuration of IPv6 addresses. SLAAC requires a /64, which is why people say, for IPv6, that a /64 is expected and that less than a /64 is clueless. However, it remains possible to hand configure a single static IPv6 address, as we are doing here.

What if, for whatever reason, we simply do not want to use SLAAC? What if our provider doesn't receive enough IPv6 addresses from his provider to allow passing on to each VPS its own /64? What if our provider's provider charges an extra fee for extra IPv6 addresses, but we do not want to pay our provider's pass through of his provider's extra fee? What if we simply choose to use single, static IPs as is traditional for servers?

• No Cloud-Init

As mentioned in the previous post of this series, most VM network setups these days are done with Cloud-Init. Proxmox supports Cloud-Init, which enables both networking and ssh access to virtual machines to be set up on the Proxmox hypervisor and outside of the VM. Cloud-init can use DHCP. Here, however, we have chosen the simplest possible manual configuration with static IPs.

• Our Static, Routed Configuration And Out of Band Gateway From Our Provider's Provider

Here, our single, static IPv4 and single, static IPv6 are each derived from a routed subnet assigned to our server node. However, our internet gateway IPv4 address is not included among our server's routed group of IPv4s. This is called an "out of band" gateway.

Besides routed subnets, it also is possible for a datacenter to assign to servers non-routed, individual IP addresses. Data for these non-routed IPs moves between the datacenter switch and server nodes via the "link layer." Hetzner has a tutorial on Debian network configuration which includes discussion of "bridged configuration" for non-routed IPs.

• Systemd in Debian Networking

Since about 2014, networking is setup on Debian with systemd. The choice of systemd initially was and has continued to be divisive. Nevertheless systemd has remained as the Debian default.

There are at least two basic variations of Debian's systemd network arrangement. The first--which seems to be the default variation for Debian systemd network configuration--at least with the netinst iso--is using systemd's networking.service. For example, by using systemctl, we can confirm that networking.service is what is being used on our Node:

root@Proxmox-VE ~ # systemctl status networking.service
● networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: 
   Active: active (exited) since Wed 2021-06-02 19:13:13 UTC; 1 weeks 2 days ago
     Docs: man:interfaces(5)
 Main PID: 791 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 4915)
   Memory: 0B
   CGroup: /system.slice/networking.service

 [ . . . ]
root@Proxmox-VE ~ # 

Our test KVM also seems to think its networking is controlled by systemd:

root@debian-kvm:~# systemctl status networking
● networking.service - Raise network interfaces
   Loaded: loaded (/lib/systemd/system/networking.service; enabled; vendor preset: enabled)
   Active: active (exited) since Wed 2021-06-16 01:20:45 UTC; 4min 51s ago
     Docs: man:interfaces(5)
  Process: 448 ExecStart=/sbin/ifup -a --read-environment (code=exited, status=0/SUCCESS)
 Main PID: 448 (code=exited, status=0/SUCCESS)

Jun 16 01:20:45 debian-kvm systemd[1]: Starting Raise network interfaces...
Jun 16 01:20:45 debian-kvm systemd[1]: Started Raise network interfaces.
root@debian-kvm:~#

As we can see, systemd networking.service calls the traditional debian ifup and ifdown.

root@debian-kvm:~# cat /lib/systemd/system/networking.service
[Unit]
Description=Raise network interfaces
Documentation=man:interfaces(5)
DefaultDependencies=no
Requires=ifupdown-pre.service
Wants=network.target
After=local-fs.target network-pre.target apparmor.service systemd-sysctl.service systemd-modules-load.service ifupdown-pre.service
Before=network.target shutdown.target network-online.target
Conflicts=shutdown.target

[Install]
WantedBy=multi-user.target
WantedBy=network-online.target

[Service]
Type=oneshot
EnvironmentFile=-/etc/default/networking
ExecStart=/sbin/ifup -a --read-environment
ExecStop=/sbin/ifdown -a --read-environment --exclude=lo
RemainAfterExit=true
TimeoutStartSec=5min
root@debian-kvm:~# 

The second Debian systemd possibility--not the default on Debian netinst.iso and not used here--is systemd-networkd. Sahitya Maruvada has a simple, clear, Debian systemd-networkd introduction, Working with systemd-networkd. The systemd-networkd wiki page and the systemd.network manpage also are available.

• Official Debian Network Setup Instructions

Official Debian network setup instructions include the Wiki, the Handbook, manual pages such as man interfaces, /etc/network/interfaces examples online, and sometimes locally:

# less /usr/share/doc/ifupdown/examples/network-interfaces

• The ip Command Usually Is Available Even Though Networking Setup Varies Among Linux Distributions

Setting up networking, DNS name resolution, and software package management is very different in different Linux distributions. Therefore, we should not assume that the steps taken below would be exactly the same with a different Linux distribution than Debian.

Nevertheless, despite the different distributions' differing network setup systems, the ip command, supplied by the iproute2 collection, usually is available these days. Please see also Red Hat's IP Command Cheat Sheet

Because the ip command often is available, networking can be configured in many distributions, including Debian, by running a sequence of ip commands. The net effect of the sequence of ip commands can be to get the network functioning on most distributions without touching that individual distribution's network setup scheme.

Here's an example of the ip command used in the context of an iPXE boot. Note that the first command in the linked example requires knowledge of the name of the interface. We can list the names of the interfaces on our system by running the ip link show command.

One issue with using a sequence of ip commands is that the network setup fails to persist across reboots. However, we can place the ip command sequence inside a script which will be run automagically every time the server reboots. The sequence of ip commands in a script reminds us of the days before systemd, when scripts controlled all parts of the boot process including network setup.

Our KVM VPS's internal network configuration that we will be using below is similar to how LXC containers are configured in Proxmox. As will be seen below, Proxmox's LXC containers' network configuration adopts a variant of the "scripted ip command" approach, which also works inside Proxmox's KVM VPSes.

V. Our VM's Network Setup

• Interfaces

Our original /etc/network/interfaces file, the one installed by the netinst.iso, might look like this:

debian@debian-kvm:~$ cd /etc/network
debian@debian-kvm:/etc/network$ cat interfaces.original
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback
debian@debian-kvm:/etc/network$ 

Note that, in the default from the netinst.iso, /etc/network/interfaces.d is empty, so sourcing its files does nothing to the configuration.

debian@debian-kvm:/etc/network$ ls interfaces.d
debian@debian-kvm:/etc/network$ 

Now, let's edit /etc/network/interfaces to match our example network information from the above Before We Start section.

debian@debian-kvm:/etc/network$ cat interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
  address 172.16.165.97/28
  gateway 172.16.164.1

iface ens18 inet6 static
  address fe80:xxxx:xxxx:xxxx::97/128
  gateway fe80:xxxx:xxxx:xxxx::3

debian@debian-kvm:/etc/network$ 

The minimum required information does not include comments (lines beginning with #). Maybe we can make the rash and short-sighted assumption that we are not going to install anything which will want a file included from interfaces.d. The loopback interface might no longer be required (please see lines 17 and 18 in this file from Debian sources). Thus, for our example setup, the minimum /etc/network/interfaces might be:

debian@debian-kvm:/etc/network$ cat interfaces

auto ens18
iface ens18 inet static
  address 172.16.165.97/28
  gateway 172.16.164.1

iface ens18 inet6 static
  address fe80:xxxx:xxxx:xxxx::97/128
  gateway fe80:xxxx:xxxx:xxxx::3

debian@debian-kvm:/etc/network$ 

When configuring Debian LXC containers, Proxmox configures their /etc/network/interfaces files using added post-up and pre-down routes. Similarly, just for fun, instead of giving the gateway addresses in our /etc/network/interfaces,, we can manually add routes. Except for the initial post-up and pre-down these added lines mirror ip route commands that we could run manually to set up or take down networking without touching the /etc/network/interfaces file.

debian@debian-kvm:/etc/network$ cat interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
  address 172.16.165.97/28
     post-up ip route add 172.16.164.1 dev ens18
     post-up ip route add default via 172.16.164.1 dev ens18
     pre-down ip route del default via 172.16.164.1 dev ens18
     pre-down ip route del 172.16.164.1 dev ens18

iface ens18 inet6 static
  address fe80:xxxx:xxxx:xxxx::97/128
     post-up ip route add fe80:xxxx:xxxx:xxxx::3  dev ens18
     post-up ip route add default via fe80:xxxx:xxxx:xxxx::3  dev ens18
     pre-down ip route del default via fe80:xxxx:xxxx:xxxx::3  dev ens18
     pre-down ip route del fe80:xxxx:xxxx:xxxx::3  dev ens18

debian@debian-kvm:/etc/network$ 

VI. Our VM's DNS

We might want to add more or different nameservers to /etc/resolv.conf. Our Quick Setup configuration, above, includes IPs from Cloudflare and from Google.

VII. Our VM's Apt Setup

The Debian wiki instructions for configuring apt are at https://wiki.debian.org/SourcesList. There also is a man page. The configuration shown above, in Section III Quick Setup, is from the SourcesList Debian wiki page.

The Debian Security Information page says:

You can use apt to easily get the latest security updates. This requires a line such as
deb http://security.debian.org/debian-security buster/updates main contrib non-free

Many of the larger providers offer Debian mirrors. For example, Debian packages and security updates are available from the Hetzner Debian Mirror

After /etc/sources.list is edited, we update our system's package repositories as follows:

apt-get upgrade && apt-get dist-upgrade -y

We can see exactly which packages are installed by looking at the logs in /var/log/apt.

We may wish to install openssh-server so that we can connect to our VM via ssh in addition to our Proxmox VNC connection. With ssh we regain cut and paste functionality while enjoying lower apparent latency!

apt-get install openssh-server

The Kennedy article, mentioned below in Section VII, has some good tips for ssh server configuration.

VIII. Security

Google suggests its first choice among essential server security articles. This article from 2013, by Bryan Kennedy, seems to provide still-good advice, except that, nowadays, many people prefer to use ed25519 keys

IX. Backup

After all this work, we certainly want to make an offline backup of our new VM. We can use Proxmox to make the backup and then download a a copy from the host node's /var/lib/vz/dump directory.

**As check with our internal team, the node is not overloaded. We would like to emphasize that as you are using VPS, everything is shared including network. As we check on our end the network is currently stable. Yes previously it was not due to some difficulties that we face but its already resolved. We would also like to emphasize that not to all speedtest server customer will be getting high speedtest result as it also depending on the geo location and the speedtest server. There are various reason that influence the speedtest result.

We only able to give customer 100M as per what customer subscribed.We check all over the network and the node, there is no indication of customer facing any network issue or instability for now. If you which to get better and stable speed, we suggest you to subscribe to Dedicated Server with dedicated bandwidth.

As for now, we do not see any further issue on this as we tested on our infra and node as well.
**

I would like to know how about all of your opinions,

Is this considered as normal or not?

Sorry for the bad english.

Thanks

Below is some of the speedtest result showing of all result is slow at Download except upload.
And only a few speedtest are stable.

Microsoft Windows [Version 10.0.19041.450]
(c) 2020 Microsoft Corporation. All rights reserved.

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 2054
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Viewqwest Pte Ltd (Singapore) [976.01 km]: 12.345 ms
Testing download speed................................................................................
Download: 388.86 Mbit/s
Testing upload speed................................................................................................
Upload: 383.68 Mbit/s
Share results: http://www.speedtest.net/result/11167017177.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 367
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by NewMedia Express (Singapore) [976.01 km]: 9.965 ms
Testing download speed................................................................................
Download: 52.27 Mbit/s
Testing upload speed................................................................................................
Upload: 353.71 Mbit/s
Share results: http://www.speedtest.net/result/11167019671.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 20637
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by OVH Cloud (Singapore) [976.01 km]: 16.871 ms
Testing download speed................................................................................
Download: 73.64 Mbit/s
Testing upload speed................................................................................................
Upload: 296.35 Mbit/s
Share results: http://www.speedtest.net/result/11167022346.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 28640
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Webe Digital Sdn Bhd (Iskandar Puteri) [992.94 km]: 11.305 ms
Testing download speed................................................................................
Download: 68.13 Mbit/s
Testing upload speed................................................................................................
Upload: 293.72 Mbit/s
Share results: http://www.speedtest.net/result/11167024248.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 19302
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Telekom Malaysia Berhad (Cyberjaya) [1205.36 km]: 12.08 ms
Testing download speed................................................................................
Download: 70.77 Mbit/s
Testing upload speed................................................................................................
Upload: 262.59 Mbit/s
Share results: http://www.speedtest.net/result/11167026070.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 5721
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Celcom Axiata (Petaling Jaya) [1206.77 km]: 6.8 ms
Testing download speed................................................................................
Download: 152.18 Mbit/s
Testing upload speed................................................................................................
Upload: 318.31 Mbit/s
Share results: http://www.speedtest.net/result/11167027728.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 19318
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Telekom Malaysia Berhad (Kuala Lumpur) [1202.85 km]: 12.684 ms
Testing download speed................................................................................
Download: 61.71 Mbit/s
Testing upload speed................................................................................................
Upload: 308.00 Mbit/s
Share results: http://www.speedtest.net/result/11167029863.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 1701
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Yes 4G (Kuala Lumpur) [1202.85 km]: 7.08 ms
Testing download speed................................................................................
Download: 288.00 Mbit/s
Testing upload speed................................................................................................
Upload: 385.21 Mbit/s
Share results: http://www.speedtest.net/result/11167031357.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 11557
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by U Mobile (Kuala Lumpur) [1202.85 km]: 6.114 ms
Testing download speed................................................................................
Download: 350.32 Mbit/s
Testing upload speed................................................................................................
Upload: 335.45 Mbit/s
Share results: http://www.speedtest.net/result/11167033662.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 8700
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by YTL Broadband (Kuala Lumpur) [1202.85 km]: 5.497 ms
Testing download speed................................................................................
Download: 251.29 Mbit/s
Testing upload speed................................................................................................
Upload: 310.45 Mbit/s
Share results: http://www.speedtest.net/result/11167035500.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 4348
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Exabytes Cloud Sdn Bhd (Kuala Lumpur) [1202.85 km]: 5.994 ms
Testing download speed................................................................................
Download: 105.04 Mbit/s
Testing upload speed................................................................................................
Upload: 306.99 Mbit/s
Share results: http://www.speedtest.net/result/11167037370.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 26511
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Maxis (Subang Jaya) [1213.83 km]: 6.152 ms
Testing download speed................................................................................
Download: 137.16 Mbit/s
Testing upload speed................................................................................................
Upload: 416.72 Mbit/s
Share results: http://www.speedtest.net/result/11167039056.png

C:\Users\administrator3\Desktop>speedtest-cli.py --share --server 28974
Retrieving speedtest.net configuration...
Testing from Shinjiru Technology Sdn Bhd (ip)...
Retrieving speedtest.net server list...
Retrieving information for the selected server...
Hosted by Digi Malaysia (Shah Alam) [1219.71 km]: 6.694 ms
Testing download speed................................................................................
Download: 385.66 Mbit/s
Testing upload speed................................................................................................
Upload: 250.08 Mbit/s
Share results: http://www.speedtest.net/result/11167040728.png

`

Hello,

If you use our Shared SQL services you must make changes to your service! Please continue reading this email!

We’re happy to announce the availability of isolated Private Networking LANs at BuyVM! These networks are isolated to you and your KVM Slice instances and allows you to use any IP addresses you want over the interface, run multicast services (for “keepalived” or services like that), or even run your own DHCP servers for VPNs. This feature also means you can provision a KVM Slice and treat it as a router, NAT’ing traffic to other servers on your LAN.

LANs are currently limited to services within the same location and don’t span locations. This might change in the future but as of now we’re keeping things simple. Private Networking is disabled by default and requires you follow the following steps to enable it.

1) Login to Stallion at https://manage.buyvm.net/
2) Click the instance you want to enable it on
3) Click Networking
4) Click “Private Networking”
5) Click the “On” button on the right
6) Click “Save Changes” at the bottom
7) Power off and Power on your service

There is no DHCP in place to automagically assign your LAN IP address. You must manually configure your choice of IP subnet. We recommend using a block from 192.168.0.0/16, 172.16.0.0/12, or 10.0.0.0/8, as these are reserved for private usage.

If you’re using our Shared SQL services you must update your IP from 172.16.0.51 to one of the following:

Las Vegas -> 205.185.112.51
New York -> 199.195.255.51

You will also have to update your “Hosts” list inside of DirectAdmin to allow your public IP address.

Please don’t hesitate to open a support ticket if you need assistance with any of this!

We hope you and your loved ones are safe & healthy throughout the holiday season.

As always, we thank you for your patronage over the years.

Francisco

add tunnel "sit0" failed: No such device

I've got a web server which needs more apps, and nmap scans don't show things which are static but powered off. Thus, I'm going to stand up an IPAM.

I've used RackTables in the past, but it's more then I want.

Suggestions?

How would one investigate and determine the cause of latency differences like this? Thanks!

First time post here but been lurking for quite a while since 2015 on "the old world". This particular issue has stumped me and I'd like to try and pick someone's brains for some help.

Here's my setup: I have two IPs being used; one for the Debian Linux system in question on the bare metal hardware, and a second IP used by a Windows Server VM running in KVM with libvirtd.

I recently switched over from one dedi to another. On my old dedi, I have a network config that looks like this:

auto br0
iface br0 inet static
address 203.111.0.114
netmask 255.255.255.248
gateway 203.111.0.113
bridge_ports eth0
bridge_stp off
bridge_maxwait 5
dns-nameservers 1.1.1.1 9.9.9.9 74.82.42.42
dns-search example.com

I then set another IP, 203.111.0.115 manually in the VM itself. This works great on this old dedi!

However a newer dedi I am moving to, with the same version of Debian Linux installed, has a slightly different setup. With the old dedi, I get a /29, but this new one, I only get 2 IPs out of a shared /24. So, my network config now looks like this on the new dedi:

auto br0
iface br0 inet static
address 203.111.0.214
netmask 255.255.255.0
gateway 203.111.0.1
bridge_ports eth0
bridge_stp off
bridge_maxwait 5
dns-nameservers 1.1.1.1 9.9.9.9 74.82.42.42
dns-search example.com

I then set a static IP on the VM just like before. Both systems can ping the default gateway at 203.111.0.1, but cannot reach anything beyond it. The Linux system's route table shows 203.111.0.1 as a default route, but is unable to pass any traffic beyond it.

I've made sure that I've gotten things like the bridge utils package installed on both systems.

Now here's the real wrinkle:

If I take out the bridge component on the new dedi, and just set the interface up without the bridge, like so:

auto eth0
allow-hotplug eth0
iface eth0 inet static
address 203.111.0.214
netmask 255.255.255.0
gateway 203.111.0.1
bridge_ports eth0
bridge_stp off
bridge_maxwait 5
dns-nameservers 1.1.1.1 9.9.9.9 74.82.42.42
dns-search example.com

This works perfectly fine and I can get out to internet destinations, as intended.

I've opened a ticket with the provider and they pointed to a issue in my network config, so I'm kind of stumped. I don't know what I'm missing here so I'm hoping someone could point me in the right direction. Thank you all in advance!