When sharing/posting YABS results, please use the proper code tag/formatting (``) just so that the results are displayed neatly.

In addition to the above, it would be very much appreciated if you were to include details such as the Provider, Plan, Location and Price. Other details like date of purchase can be included as well, or just indicate if it's a deal bought on a special occasion, i.e., Black Friday, Cyber Monday, Independence Day etc.

If you're posting results of a custom built machine, or something locally of your own, it would be nice if you could share details of your build and/or hardware - just to serve as an "FYI" for the community members.

Here's an example of the format you can use to share your YABS results:

# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
#              Yet-Another-Bench-Script              #
#                     v2022-06-11                    #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #

Mon 04 Jul 2022 04:28:07 PM EDT

Basic System Information:
---------------------------------
Uptime     : 0 days, 0 hours, 6 minutes
Processor  : Intel(R) Xeon(R) Gold 5315Y CPU @ 3.20GHz
CPU cores  : 1 @ 3199.998 MHz
AES-NI     : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM        : 1.9 GiB
Swap       : 975.0 MiB
Disk       : 28.4 GiB
Distro     : Debian GNU/Linux 11 (bullseye)
Kernel     : 5.10.0-15-amd64

fio Disk Speed Tests (Mixed R/W 50/50):
---------------------------------
Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 264.91 MB/s  (66.2k) | 3.46 GB/s    (54.1k)
Write      | 265.61 MB/s  (66.4k) | 3.48 GB/s    (54.4k)
Total      | 530.52 MB/s (132.6k) | 6.95 GB/s   (108.6k)
           |                      |
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ----
Read       | 9.32 GB/s    (18.2k) | 7.95 GB/s     (7.7k)
Write      | 9.82 GB/s    (19.1k) | 8.48 GB/s     (8.2k)
Total      | 19.15 GB/s   (37.4k) | 16.44 GB/s   (16.0k)

iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed
                |                           |                 |
Clouvider       | London, UK (10G)          | 828 Mbits/sec   | 469 Mbits/sec
Online.net      | Paris, FR (10G)           | 882 Mbits/sec   | 262 Mbits/sec
Hybula          | The Netherlands (40G)     | 882 Mbits/sec   | 823 Mbits/sec
Uztelecom       | Tashkent, UZ (10G)        | 577 Mbits/sec   | 307 Mbits/sec
Clouvider       | NYC, NY, US (10G)         | 940 Mbits/sec   | 939 Mbits/sec
Clouvider       | Dallas, TX, US (10G)      | 746 Mbits/sec   | 155 Mbits/sec
Clouvider       | Los Angeles, CA, US (10G) | 796 Mbits/sec   | 619 Mbits/sec

iperf3 Network Speed Tests (IPv6):
---------------------------------
Provider        | Location (Link)           | Send Speed      | Recv Speed
                |                           |                 |
Clouvider       | London, UK (10G)          | 705 Mbits/sec   | 435 Mbits/sec
Online.net      | Paris, FR (10G)           | 823 Mbits/sec   | 431 Mbits/sec
Hybula          | The Netherlands (40G)     | 714 Mbits/sec   | 725 Mbits/sec
Uztelecom       | Tashkent, UZ (10G)        | 676 Mbits/sec   | 410 Mbits/sec
Clouvider       | NYC, NY, US (10G)         | 926 Mbits/sec   | 926 Mbits/sec
Clouvider       | Dallas, TX, US (10G)      | 904 Mbits/sec   | 775 Mbits/sec
Clouvider       | Los Angeles, CA, US (10G) | 885 Mbits/sec   | 632 Mbits/sec

Geekbench 5 Benchmark Test:
---------------------------------
Test            | Value
                |
Single Core     | 1036
Multi Core      | 1005
Full Test       | https://browser.geekbench.com/v5/cpu/15825267

Thank you and happy YABS-ing!

All information can be found on https://network-speed.xyz/

Test Locations
* North America
* South America
* Europe
* Asia
* Africa
* GCC Middle East
* India
* China
* Iran
* More coming soon!

Usage
For Global Test

wget -qO- network-speed.xyz | bash

For Regional Test

curl -sL network-speed.xyz | bash -s -- -r region_name

region_name = na, sa, eu, asia, Africa, middle-east, india, china, iran

Global Test Sample Output

---------------------------- network-speed.xyz ----------------------------
      A simple script to test network performance using speedtest-cli
---------------------------------------------------------------------------
 Version            : v2023.04.25
 Global Speedtest   : wget -qO- network-speed.xyz | bash
 Region Speedtest   : wget -qO- network-speed.xyz | bash -s -- -r <region>
---------------------------------------------------------------------------
 Basic System Info
---------------------------------------------------------------------------
 CPU Model          : AMD Ryzen 9 3900X 12-Core Processor
 CPU Cores          : 1 @ 3800.000 MHz
 CPU Cache          : 512 KB
 AES-NI             : ✔ Enabled
 VM-x/AMD-V         : ✔ Enabled
 Total Disk         : 24.3 GB (6.9 GB Used)
 Total RAM          : 1.1 GB (210.2 MB Used)
 System uptime      : 4 days, 12 hour 38 min
 Load average       : 0.07, 0.04, 0.03
 OS                 : Ubuntu 20.04.6 LTS
 Arch               : x86_64 (64 Bit)
 Kernel             : 5.4.0-147-generic
 Virtualization     : KVM
---------------------------------------------------------------------------
 Basic Network Info
---------------------------------------------------------------------------
 Primary Network    : IPv4
 ISP                : Alibaba.com LLC
 ASN                : AS35913 DediPath
 Host               : Zenlayer Inc, Virtual Machine Solutions LLC
 Location           : Seattle, Washington-WA, United States
---------------------------------------------------------------------------
 Speedtest.net (Region: GLOBAL)
---------------------------------------------------------------------------
 Location         Latency     Loss    DL Speed       UP Speed       Server

 ISP: DediPath

 Nearest          39.10 ms    0.0%    874.86 Mbps    773.67 Mbps    Ideatek Telcom - Wichita, KS

 Kochi, IN        279.63 ms   0.0%    505.94 Mbps    252.15 Mbps    Asianet Broadband - Cochin
 Bangalore, IN    251.37 ms   0.3%    645.52 Mbps    269.74 Mbps    Bharti Airtel Ltd - Bangalore
 Chennai, IN      230.78 ms   N/A     551.27 Mbps    354.59 Mbps    Jio - Chennai
 Mumbai, IN       289.37 ms   0.3%    366.12 Mbps    116.30 Mbps    i3D.net - Mumbai
 Delhi, IN        267.42 ms   0.0%    596.24 Mbps    159.85 Mbps    Tata Play Fiber - New Delhi

 Los Angeles, US  28.41 ms    0.0%    884.35 Mbps    533.28 Mbps    ReliableSite Hosting - Los Angeles, CA
 Dallas, US       53.31 ms    0.0%    774.85 Mbps    452.48 Mbps    Hivelocity - Dallas, TX
 New York, US     80.01 ms    0.0%    315.54 Mbps    797.42 Mbps    Clouvider Ltd - New York, NY
 Seattle, US      2.37 ms     N/A     938.03 Mbps    935.63 Mbps    Comcast - Seattle, WA
 Miami, US        89.86 ms    0.0%    823.65 Mbps    660.29 Mbps    AT&T - Miami, FL
 Toronto, CA      66.14 ms    0.0%    724.23 Mbps    618.85 Mbps    Rogers - Toronto, ON

 Paris, FR        158.80 ms   0.0%    675.62 Mbps    553.16 Mbps    ORANGE FRANCE - Paris
 Amsterdam, NL    157.07 ms   0.0%    3.55 Mbps      460.41 Mbps    Clouvider Ltd - Amsterdam
 Warsaw, PL       178.59 ms   0.0%    717.62 Mbps    411.57 Mbps    UPC Polska - Warszawa
 London, UK       146.49 ms   0.0%    857.45 Mbps    37.09 Mbps     VeloxServ Communications - London
 Frankfurt, DE    173.11 ms   0.0%    797.18 Mbps    407.43 Mbps    23M GmbH - Frankfurt Am Main

 Dubai, AE        281.79 ms   0.0%    517.34 Mbps    291.43 Mbps    du - Dubai
 Fujairah, AE     247.51 ms   0.0%    543.98 Mbps    296.35 Mbps    ETISALAT-UAE - Fujairah
 Jeddah, KSA      211.72 ms   0.0%    621.52 Mbps    407.56 Mbps    Saudi Telecom Company

 Shanghai, CU-CN  200.57 ms   0.0%    688.99 Mbps    307.36 Mbps    China Unicom 5G - ShangHai
 Nanjing, CT-CN   168.96 ms   0.3%    659.64 Mbps    461.14 Mbps    China Telecom JiangSu 5G - Nanjing
 Hong Kong, HKG   157.84 ms   N/A     787.04 Mbps    297.71 Mbps    STC - Hong Kong
 Singapore, SG    174.90 ms   0.0%    700.25 Mbps    343.45 Mbps    i3D.net - Singapore
 Jakarta, ID      244.22 ms   0.0%    412.39 Mbps    172.91 Mbps    PT Indosat Tbk - Jakarta
 Tokyo, JP        111.60 ms   N/A     645.79 Mbps    433.46 Mbps    fdcservers.net - Tokyo
---------------------------------------------------------------------------
 Avg DL Speed       : 639.57 Mbps
 Avg UL Speed       : 415.59 Mbps

 Total DL Data      : 21.90 GB
 Total UL Data      : 14.15 GB
 Total Data         : 36.05 GB
---------------------------------------------------------------------------
 Duration           : 13 min 5 sec
 System Time        : 27/04/2023 - 01:10:33 IST
 Total Script Runs  : 1063
---------------------------------------------------------------------------
 Result             : https://cdn1.frocdn.ch/JrA3CugVFKYukMj.txt
---------------------------------------------------------------------------
root@us9-virmach-seattle:~#

Regional Test Sample Output
NA: https://cdn1.frocdn.ch/o0jvHlZr9cUQSa2.txt
SA: https://cdn1.frocdn.ch/t31SkeDTWhuwbHH.txt
EU: https://cdn1.frocdn.ch/UzgzCEzBqFzAOVf.txt
Asia: https://rentry.co/q7qvg
GCC: https://cdn1.frocdn.ch/5F5jacWigHcsMQu.txt
India: https://rentry.co/q7qvg
China: https://pastebin.com/4ZiB92m4
Iran: https://pastebin.com/LuFsGM6H

Of course, I am actively developing and updating this regularly, so any feedbacks and suggestions are welcome!

Today I wanted to present you, yammdb.
Which is another, different, geodatabase, based on real measurements.

You can find it here: https://github.com/Ne00n/yammdb
The Database is build weekly, on fridays.

As long the buildserver doesn't blow up.
The primary use case for me is to compare existing geo data, maybe someone of you will find it useful.

If you have any ideas and feedback, please lemme know.

Enjoy.

Today I first heard about systemd-nspawn when andrewstuart mentioned it on HN.

Doesn't look like there has been much mention of nspawn here on LES, but there is a Debian wiki page and an Arch wiki page. There is a man page on FreeDesktop.org.

Quoting from the Arch wiki: "systemd-nspawn may be used to run a command or OS in a light-weight namespace container. . . . systemd-nspawn is a simpler tool to configure than LXC or Libvirt." Hmm, simple configuration sounds pretty good!

There seems to be an nspawn.org website, which talks about a wrapper script for easier install, and there is a Github repo as well.

Might a server with a bunch of nspawn containers create an easily configured Low End Empire? 💰

Does anybody here have experience using nspawn?

In December, I decided to turn some of them into something useful (hopefully...) for the community and dnscry.pt was born:

Public DNSCrypt resolvers hosted by LowEnd providers

Most of the servers have been taken from my collection of idlers, but I'd like to give a shout-out to @Kuroit and @terrahost who are generously sponsoring three servers for the project.

In a nutshell, DNSCrypt is a protocol which encrypt and authenticates your DNS requests, so that a third party (like your ISP) can no longer tinker with them. You have to run a DNSCrypt client like dnscrypt-proxy locally or in your network and point your DNS requests there instead of towards your Wi-Fi router or public resolvers like Google's 8.8.8.8. Your DNSCrypt client will take care of the encryption and forward your requests to a public DNSCrypt resolver (like one of those I run for dnscry.pt).

None of the resolvers do any filtering of any kind. I don't store any logs of your requests. All I do is collect metrics using Munin.

If you're interested in giving it a try, further instructions can be found here. There's also a list of all resolvers.

Singapore may appear offline from time to time but is doing fine for local traffic. Intercontinental traffic is going through Cogent and their lines appear to be congested af. I'm monitoring all resolvers from a server hosted in Jacksonville, FL.

If you're using dnscrypt-proxy, you don't have to handpick resolvers near your location. Instead, use the auto-generated resolver list. Configuration instructions can be found in the file header or in the "Get Started" guide on the website.

I hope this is useful for some of you. I'm using DNSCrypt for years and have switched to my resolver list recently.

Let me know if you have any feedback or questions. I'm also open for suggestions for new locations.

Alert Info
Node: 12-eu-adv
Chart ID: ipv4.tcphandshake
Type: System
Component: Network
Class: Errors
Check every: 10 seconds
10s_ipv4_tcp_resets_sent
Triggered 6/2/23 15:02:55 0.0322 tcp resets/s
Alert Description- average number of sent TCP RESETS over the last 10 seconds. This can indicate a port scan, or that a service running on this host has crashed.

I added a LAN IP address to my WireGuard server like this:

ip addr add 10.10.10.123/32 dev wg0

I happened not to use this IP for anything. The IP was pingable when connected to WireGuard, the server would respond.

The hosting provider suspended my server because of IP spoofing, that someone had been using that unassigned IP to spoof my public IP.

This was fixed easily, but now I have questions.

How is that possible? Given that there was no breach, it means that the unassigned LAN IP could be somehow used to act as the external IP?

Below i attached the ss of current server load.

root@rescue ~ # cat start-install 
qemu-system-x86_64 \
    -enable-kvm \
    -nographic \
    -cpu host \
    -boot once=d \
    -cdrom ./boot-com.iso \
    -m 4096 \
    -drive file=/dev/nvme0n1,format=raw,media=disk,if=virtio \
    -drive file=/dev/nvme1n1,format=raw,media=disk,if=virtio 
root@rescue ~ # 

One downloads from the above linked page the SHA512 file and the boot-com.iso file, checks the SHA512, and runs something like the above qemu-command. A prerequisite might be having used Hetzner installimage to put a non-RAID install on the server so that the rescue system's qemu will see /dev/nvme* devices instead of /dev/md*.

After running through the install sequence and rebooting inside qemu inside tmux, the system seemed to come up from the NVMe drive.

After shutdown -h now inside NetBSD and then Ctrl-b, x, y to get out of tmux and then rebooting to get out of the Hetzner Rescue System, the server seemed to come up! It responded to IPv4 ping, but ssh root@fsn.metalvps.com gave me ssh: connect to host fsn.metalvps.com port 22: Network is unreachable.

Re-enabling the Hetzner rescue system and restarting qemu, NetBSD booted again from the NVMe disk (same qemu command as above, but with the following two lines removed).

     -boot once=d \
    -cdrom ./boot-com.iso \

Eventually I may figure out why the live, booted-from-metal server could respond to ping but ssh wouldn't work. It's been more than 20 years since I last used NetBSD, so I've forgotten everything.

Why didn't I use Hetzner's free KVM console option? Well, if qemu works, I don't have to bother Hetzner's support guys to have them attach and then remove the KVM.

Thanks to the depenguin.me guys for helpful hints.

Here's ps aux output from the rebooted server running inside qemu. Notice that there are fewer processes than contemporary Linux.

fsn# date -u; ps aux
Tue May 16 23:57:19 UTC 2023
USER     PID %CPU %MEM   VSZ   RSS TTY   STAT STARTED    TIME COMMAND
root       0  0.0  0.8     0 33688 ?     DKl  11:30PM 0:00.32 [system]
root       1  0.0  0.0 11944  1660 ?     Is   11:30PM 0:00.00 init 
postfix  328  0.0  0.1 21456  4900 ?     I    11:30PM 0:00.01 pickup -l -t unix
postfix  329  0.0  0.1 21504  4968 ?     I    11:30PM 0:00.00 qmgr -l -t unix -
root     331  0.0  0.0 13360  1488 ?     Is   11:30PM 0:00.00 /usr/sbin/inetd -
root     339  0.0  0.0 12212  1740 ?     Is   11:30PM 0:00.00 /usr/sbin/cron 
root     351  0.0  0.1 18408  2676 ?     S    11:30PM 0:00.03 ntpd: asynchronou
root     829  0.0  0.1 18036  2476 ?     Ss   11:30PM 0:00.03 /usr/sbin/syslogd
root    1313  0.0  0.0 11768  1416 ?     Is   11:30PM 0:00.00 /usr/sbin/powerd 
root    1442  0.0  0.4 18272 17824 ?     Ss   11:30PM 0:00.14 /usr/sbin/ntpd -p
root    1532  0.0  0.1 23044  3200 ?     Is   11:30PM 0:00.00 sshd: /usr/sbin/s
root    1821  0.0  0.1 21528  2876 ?     Is   11:30PM 0:00.01 /usr/libexec/post
root     306  0.0  0.0 12076  1720 tty00 O+   11:57PM 0:00.00 ps -aux 
root     335  0.0  0.1 12584  2228 tty00 S    11:31PM 0:00.07 -sh 
root     347  0.0  0.3 24872 10492 tty00 Is   11:30PM 0:00.09 login 
root     348  0.0  0.0 11860  1684 ttyE1 Is+  11:30PM 0:00.00 /usr/libexec/gett
root    2013  0.0  0.0 11856  1684 ttyE2 Is+  11:30PM 0:00.00 /usr/libexec/gett
root    1118  0.0  0.0 11856  1680 ttyE3 Is+  11:30PM 0:00.00 /usr/libexec/gett
fsn# 

Here's dmesg output from the rebooted server running inside qemu.

fsn# dmesg | cat -n                                                                                                                                        
     1  [     1.000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,                                                                      
     2  [     1.000000]     2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,                                                                    
     3  [     1.000000]     2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023                                                                     
     4  [     1.000000]     The NetBSD Foundation, Inc.  All rights reserved.                                                                              
     5  [     1.000000] Copyright (c) 1982, 1986, 1989, 1991, 1993                                                                                         
     6  [     1.000000]     The Regents of the University of California.  All rights reserved.                                                             
     7                                                                                                                                                     
     8  [     1.000000] NetBSD 10.99.4 (GENERIC) #0: Sat May 13 11:19:19 UTC 2023                                                                          
     9  [     1.000000]         mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC                                                         
    10  [     1.000000] total memory = 4095 MB                                                                                                             
    11  [     1.000000] avail memory = 3935 MB                                                                                                             
    12  [     1.000000] timecounter: Timecounters tick every 10.000 msec                                                                                   
    13  [     1.000000] Kernelized RAIDframe activated                                                                                                     
    14  [     1.000000] timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100                                                                  
    15  [     1.000004] mainbus0 (root)                                                                                                                    
    16  [     1.000004] ACPI: RSDP 0x00000000000F5AF0 000014 (v00 BOCHS )                                                                                  
    17  [     1.000004] ACPI: RSDT 0x00000000BFFE1550 000034 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)                                                  
    18  [     1.000004] ACPI: FACP 0x00000000BFFE1404 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)                                                  
    19  [     1.000004] ACPI: DSDT 0x00000000BFFE0040 0013C4 (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)                                                  
    20  [     1.000004] ACPI: FACS 0x00000000BFFE0000 000040                                                                                               
    21  [     1.000004] ACPI: APIC 0x00000000BFFE1478 000078 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)                                                  
    22  [     1.000004] ACPI: HPET 0x00000000BFFE14F0 000038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)                                                  
    23  [     1.000004] ACPI: WAET 0x00000000BFFE1528 000028 (v01 BOCHS  BXPCWAET 00000001 BXPC 00000001)                                                  
    24  [     1.000004] ACPI: 1 ACPI AML tables successfully acquired and loaded                                                                           
    25  [     1.000004] ioapic0 at mainbus0 apid 0: pa 0xfec00000, version 0x11, 24 pins                                                                   
    26  [     1.000004] cpu0 at mainbus0 apid 0                                                                                                            
    27  [     1.000004] cpu0: Use lfence to serialize rdtsc                                                                                                
    28  [     1.000004] cpu0: 13th Gen Intel(R) Core(TM) i9-13900, id 0xb0671
    29  [     1.000004] cpu0: node 0, package 0, core 0, smt 0
    30  [     1.000004] acpi0 at mainbus0: Intel ACPICA 20221020
    31  [     1.000004] acpi0: X/RSDT: OemId <BOCHS ,BXPCRSDT,00000001>, AslId <BXPC,00000001>
    31  [     1.000004] acpi0: X/RSDT: OemId <BOCHS ,BXPCRSDT,00000001>, AslId <BXPC,00000001>                                                     [76/526]
    32  [     1.000004] LNKS: ACPI: Found matching pin for 0.1.INTA at func 3: 9                                                                           
    33  [     1.000004] LNKC: ACPI: Found matching pin for 0.3.INTA at func 0: 11                                                                          
    34  [     1.000004] LNKD: ACPI: Found matching pin for 0.4.INTA at func 0: 11                                                                          
    35  [     1.000004] LNKA: ACPI: Found matching pin for 0.5.INTA at func 0: 10                                                                          
    36  [     1.000004] acpi0: SCI interrupting at int 9                                                                                                   
    37  [     1.000004] acpi0: fixed power button present                                                                                                  
    38  [     1.000004] timecounter: Timecounter "ACPI-Fast" frequency 3579545 Hz quality 1000                                                             
    39  [     1.046041] hpet0 at acpi0: high precision event timer (mem 0xfed00000-0xfed00400)                                                             
    40  [     1.046041] timecounter: Timecounter "hpet0" frequency 100000000 Hz quality 2000                                                               
    41  [     1.046619] pckbc1 at acpi0 (KBD, PNP0303) (kbd port): io 0x60,0x64 irq 1                                                                      
    42  [     1.046619] pckbc2 at acpi0 (MOU, PNP0F13) (aux port): irq 12                                                                                  
    43  [     1.046619] fdc0 at acpi0 (FDC0, PNP0700): io 0x3f2-0x3f5,0x3f7 irq 6 drq 2                                                                    
    44  [     1.046619] lpt0 at acpi0 (LPT1, PNP0400-1): io 0x378-0x37f irq 7                                                                              
    45  [     1.046619] com0 at acpi0 (COM1, PNP0501-1): io 0x3f8-0x3ff irq 4                                                                              
    46  [     1.046619] com0: ns16550a, 16-byte FIFO                                                                                                       
    47  [     1.046619] com0: console                                                                                                                      
    48  [     1.046619] qemufwcfg0 at acpi0 (FWCF, QEMU0002): io 0x510-0x51b                                                                               
    49  [     1.046619] qemufwcfg0: <QEMU>                                                                                                                 
    50  [     1.046619] ACPI: Enabled 2 GPEs in block 00 to 0F                                                                                             
    51  [     1.046619] pckbd0 at pckbc1 (kbd slot)                                                                                                        
    52  [     1.046619] pckbc1: using irq 1 for kbd slot                                                                                                   
    53  [     1.046619] wskbd0 at pckbd0 mux 1                                                                                                             
    54  [     1.046619] pms0 at pckbc1 (aux slot)                                                                                                          
    55  [     1.046619] pckbc1: using irq 12 for aux slot                                                                                                  
    56  [     1.046619] wsmouse0 at pms0 mux 0                                                                                                             
    57  [     1.046619] pci0 at mainbus0 bus 0: configuration mode 1                                                                                       
    58  [     1.046619] pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok                                                                 
    59  [     1.046619] pchb0 at pci0 dev 0 function 0: Intel 82441FX (PMC) PCI and Memory Controller (rev. 0x02)                                          
    60  [     1.046619] pcib0 at pci0 dev 1 function 0: Intel 82371SB (PIIX3) PCI-ISA Bridge (rev. 0x00)                                                   
    61  [     1.046619] piixide0 at pci0 dev 1 function 1: Intel 82371SB IDE Interface (PIIX3) (rev. 0x00)                                                 
    62  [     1.046619] piixide0: bus-master DMA support present                                                                                           
    63  [     1.046619] piixide0: primary channel wired to compatibility mode                                                                              
    64  [     1.046619] piixide0: primary channel interrupting at ioapic0 pin 14                                                                           
    65  [     1.046619] atabus0 at piixide0 channel 0                                                                                                      
    66  [     1.046619] piixide0: secondary channel wired to compatibility mode                                                                            
    67  [     1.046619] piixide0: secondary channel interrupting at ioapic0 pin 15                                                                         
    68  [     1.046619] atabus1 at piixide0 channel 1 
    69  [     1.046619] piixpm0 at pci0 dev 1 function 3: Intel 82371AB (PIIX4) Power Management Controller (rev. 0x03)                            [38/526]
    70  [     1.046619] timecounter: Timecounter "piixpm0" frequency 3579545 Hz quality 1000                                                               
    71  [     1.046619] piixpm0: 24-bit timer                                                                                                              
    72  [     1.046619] piixpm0: interrupting at ioapic0 pin 9                                                                                             
    73  [     1.046619] iic0 at piixpm0 port 0: I2C bus                                                                                                    
    74  [     1.046619] vga0 at pci0 dev 2 function 0: vendor 1234 product 1111 (rev. 0x02)                                                                
    75  [     1.046619] wsdisplay0 at vga0 kbdmux 1                                                                                                        
    76  [     1.046619] wsmux1: connecting to wsdisplay0                                                                                                   
    77  [     1.046619] wskbd0: connecting to wsdisplay0                                                                                                   
    78  [     1.046619] drm at vga0 not configured                                                                                                         
    79  [     1.046619] wm0 at pci0 dev 3 function 0, 64-bit DMA: Intel i82540EM 1000BASE-T Ethernet (rev. 0x03)                                           
    80  [     1.046619] wm0: interrupting at ioapic0 pin 11                                                                                                
    81  [     1.046619] wm0: 32-bit 33MHz PCI bus                                                                                                          
    82  [     1.046619] wm0: 64 words (6 address bits) MicroWire EEPROM                                                                                    
    83  [     1.046619] wm0: Ethernet address 52:54:00:12:34:56                                                                                            
    84  [     1.046619] wm0: 0x200002<LOCK_EECD,WOL>                                                                                                       
    85  [     1.046619] makphy0 at wm0 phy 1: Marvell 88E1011 Gigabit PHY, rev. 0                                                                          
    86  [     1.046619] makphy0: Failed to access EADR. Are you an emulator?                                                                               
    87  [     1.046619] makphy0: Failed to read EXTSR. Are you an emulator?. Regard as 1000BASE-T.                                                         
    88  [     1.046619] makphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto                                            
    89  [     1.046619] virtio0 at pci0 dev 4 function 0                                                                                                   
    90  [     1.046619] virtio0: block device (id 2, rev. 0x00)                                                                                            
    91  [     1.046619] ld0 at virtio0: features: 0x10000a54<INDIRECT_DESC,CONFIG_WCE,FLUSH,BLK_SIZE,GEOMETRY,SEG_MAX>                                     
    92  [     1.046619] virtio0: allocated 1060864 byte for virtqueue 0 for I/O request, size 256                                                          
    93  [     1.046619] virtio0: using 1048576 byte (65536 entries) indirect descriptors                                                                   
    94  [     1.046619] virtio0: config interrupting at msix0 vec 0                                                                                        
    95  [     1.046619] virtio0: queues interrupting at msix0 vec 1                                                                                        
    96  [     1.046619] ld0: 1788 GB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 3750748848 sectors                                                      
    97  [     1.046619] virtio1 at pci0 dev 5 function 0                                                                                                   
    98  [     1.046619] virtio1: block device (id 2, rev. 0x00)                                                                                            
    99  [     1.046619] ld1 at virtio1: features: 0x10000a54<INDIRECT_DESC,CONFIG_WCE,FLUSH,BLK_SIZE,GEOMETRY,SEG_MAX>                                     
   100  [     1.046619] virtio1: allocated 1060864 byte for virtqueue 0 for I/O request, size 256                                                          
   101  [     1.046619] virtio1: using 1048576 byte (65536 entries) indirect descriptors                                                                   
   102  [     1.046619] virtio1: config interrupting at msix1 vec 0                                                                                        
   103  [     1.046619] virtio1: queues interrupting at msix1 vec 1                                                                                        
   104  [     1.046619] ld1: 1788 GB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 3750748848 sectors                                                      
   105  [     1.046619] isa0 at pcib0                                                                                                                      
   106  [     1.046619] attimer0 at isa0 port 0x40-0x43 
   107  [     1.046619] pcppi0 at isa0 port 0x61
   108  [     1.046619] spkr0 at pcppi0: PC Speaker
   109  [     1.046619] wsbell at spkr0 not configured
   110  [     1.046619] midi0 at pcppi0: PC speaker
   111  [     1.046619] sysbeep0 at pcppi0
   112  [     1.046619] attimer0: attached to pcppi0
   113  [     1.046619] acpicpu0 at cpu0: ACPI CPU
   114  [     1.046619] acpicpu0: C1: HLT, lat   0 us, pow     0 mW
   115  [     1.046619] vmt0 at cpu0
   116  [     1.046619] vmt0: UUID: 00000000-0000-0000-0000-000000000000
   117  [     1.046619] vmware: open failed, eax=0xffffffff, ecx=0x1e, edx=0x5658
   118  [     1.046619] vmt0: autoconfiguration error: failed to open backdoor RPC channel (TCLO protocol)
   119  [     1.046619] timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
   120  [     1.046619] fd0 at fdc0 drive 0: 1.44MB, 80 cyl, 2 head, 18 sec
   121  [     1.046619] ld1: GPT GUID: 7ac59b06-d775-499c-b672-f71133e62986
   122  [     1.046619] ld0: GPT GUID: 868d363e-b930-4142-944f-e70905997bdb
   123  [     1.046619] IPsec: Initialized Security Association Processing.
   124  [     1.046619] dk0 at ld1: "be45d1a2-c15f-4fa9-a36b-db22ab2e15e0", 524288 blocks at 4096, type: msdos
   125  [     1.046619] dk1 at ld1: "c6fc5a64-7669-464c-8578-77c52a1ebe6c", 1048576 blocks at 528384, type: <unknown>
   126  [     1.046619] dk2 at ld1: "24fde94d-ca86-46bd-8358-93c1ac2f8b9e", 3749171855 blocks at 1576960, type: <unknown>
   127  [     1.046619] dk3 at ld0: "02a153e5-14e1-45f0-8e65-d55f5f7da62c", 3742359552 blocks at 2048, type: ffs
   128  [     1.046619] dk4 at ld0: "981d2b55-1018-4fc9-9f3c-5847d6751eb7", 8387215 blocks at 3742361600, type: swap
   129  [     5.005793] atapibus0 at atabus1: 2 targets
   130  [     5.015571] cd0 at atapibus0 drive 0: <QEMU DVD-ROM, QM00003, 2.5+> cdrom removable
   131  [     5.015571] cd0: 32-bit data port
   132  [     5.015571] cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
   133  [     5.015571] cd0(piixide0:1:0): using PIO mode 4, DMA mode 2 (using DMA)
   134  [     5.015571] swwdog0: software watchdog initialized
   135  [     5.025557] WARNING: 1 error while detecting hardware; check system log.
   136  [     5.025557] boot device: ld0
   137  [     5.025557] root on dk3 dumps on dk4
   138  [     5.025557] root file system type: ffs
   139  [     5.025557] kern.module.path=/stand/amd64/10.99.4/modules
   140  [    10.485560] wsdisplay0: screen 1 added (80x25, vt100 emulation)
   141  [    10.485560] wsdisplay0: screen 2 added (80x25, vt100 emulation)
   142  [    10.485560] wsdisplay0: screen 3 added (80x25, vt100 emulation)
   143  [    10.485560] wsdisplay0: screen 4 added (80x25, vt100 emulation)
fsn# 

https://github.com/89luca89/distrobox

I glanced at this last night. I haven't read much of it, and I haven't tried it. I never heard of it, or its antecedent mentioned in the README.md file.

Could this, or something else like it, be the basis of a super fast, distro agnostic, shared server?

Thanks! Friendly greetings!

Thanks!

So is our future going to be bun.js or deno.js 2? Or something else?

I've had this question in my sig for a few days. I'm still wondering what the answer is. I'm not trying to push any particular perspective. I'm just . . . wondering. What do you think?

Best!

Tom

Which SSDs do you use in your Ceph storage?
How important is the RAM speed (I could imagine that this plays an upscale role with SSDs)?
Any good or negative experiences you can share?

I would be extremely happy if a lively exchange of experiences and opinions takes place here, without anyone attacking anyone else.

Thanks a lot for your support.

Now with falling price of server grade hardware, it's cheaper to buy. . . .

Are server prices really falling?

What's the current cost of a great deal to buy a server in the US + colo? Something with server grade hardware, but otherwise somewhat like the i9-13900 I have from Hetzner (64 GB DDR5 ECC, 2 x 1.92 TB NVMe Gen 4, 1 Gbps unlimited) for € 89.00/month?

I don't have a server in the US at the moment. A US location with good connections to Asia would be nice.

Initially when installing container IPv6 works fine, but randomly dies after a while on some of the containers.
From container -

Some containers have working IPv6 , some don't.

The host node is unaffected by this issue:

[root@dro2-nld ~]# ping6 google.com
PING google.com(ams15s41-in-x0e.1e100.net (2a00:1450:400e:802::200e)) 56 data bytes
64 bytes from ams16s21-in-x0e.1e100.net (2a00:1450:400e:802::200e): icmp_seq=1 ttl=120 time=4.74 ms
64 bytes from ams16s21-in-x0e.1e100.net (2a00:1450:400e:802::200e): icmp_seq=2 ttl=120 time=2.78 ms
64 bytes from ams16s21-in-x0e.1e100.net (2a00:1450:400e:802::200e): icmp_seq=3 ttl=120 time=3.04 ms
64 bytes from ams16s21-in-x0e.1e100.net (2a00:1450:400e:802::200e): icmp_seq=4 ttl=120 time=2.73 ms
64 bytes from ams16s21-in-x0e.1e100.net (2a00:1450:400e:802::200e): icmp_seq=5 ttl=120 time=2.82 ms
^C
--- google.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 2.734/3.225/4.742/0.767 ms

Checking it further ,

I made a test VM and assigned it IPv6 '2001:41d0:a:28b6:7cbb:0000:0000:0001'

Tried pinging the address and it was not reachable globally https://prnt.sc/vmRGmN2P0UDZ

I added that IPv6 address to the host node network interface
ip -6 addr add 2001:41d0:a:28b6:7cbb:0000:0000:0001/80 dev br0

It responds to pings now https://prnt.sc/CdgToWos2TFZ

I removed the IPv6 address from the host node network interface ip -6 addr del 2001:41d0:a:28b6:7cbb:0000:0000:0001/80 dev br0

It still responds to ping fine https://prnt.sc/3prDDpT7I_z7

The container has working IPv6 connectivity now

root@test:/# ping6 google.com
PING google.com(par21s20-in-x0e.1e100.net (2a00:1450:4007:818::200e)) 56 data bytes
64 bytes from par21s20-in-x0e.1e100.net (2a00:1450:4007:818::200e): icmp_seq=1 ttl=115 time=4.69 ms
64 bytes from par21s20-in-x0e.1e100.net (2a00:1450:4007:818::200e): icmp_seq=2 ttl=115 time=4.64 ms
64 bytes from par21s20-in-x0e.1e100.net (2a00:1450:4007:818::200e): icmp_seq=3 ttl=115 time=4.79 ms
^C
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 4.637/4.703/4.785/0.100 ms

This issue did not exist in previous openvz versions and appeared after the upgrade.

Any idea what is causing this ? I think using ndpresponder will resolve this issue, but I haven't figured how to configure it yet.

I recognised the chipset name and when I checked my Hetzner box (i9-9900k), sure enough it had one of these. However, whenever I've done a YABS, I did get around 900Mbps, so not entirely sure what's going on here. I'm using debian 11 which has kernel 5.10.0, so in theory this regression should have been evident on my system.

Simple Tutorial to create your own .mmdb for gdnsd.

You need a .mmdb writer, I suggest coded in python3, but there is also one in Perl and Go, depending on what you prefer.

Python3 writer can be found here:
https://github.com/vimt/MaxMind-DB-Writer-python
https://github.com/maxmind/mmdbwriter (Go)
https://github.com/maxmind/MaxMind-DB-Writer-perl (Perl)

I slightly needed to modify it, because of the data type, gdnsd strictly validates the geo cordinates.
https://github.com/Ne00n/MaxMind-DB-Writer-python

After Installing, you can build your own, .mmdb, that works out of the box with gdnsd.

from netaddr import IPNetwork, IPSet
from mmdb_writer import MMDBWriter
import geoip2.database

writer = MMDBWriter(4, 'GeoIP2-City', languages=['EN'], description="Mah own .mmdb")
subnets = ["1.1.1.0/24","8.8.8.0/24"]
writer.insert_network(IPSet(subnets), {'location':{"latitude":1.11,"longitude":2.22}})
writer.to_db_file('geo.mmdb')

reader = geoip2.database.Reader("geo.mmdb")

response = reader.city("1.1.1.1")
print(response.location.latitude,response.location.longitude)

response = reader.city("8.8.8.8")
print(response.location.latitude,response.location.longitude)

Will return.

1.11 2.22
1.11 2.22

Put the geo.mmdb into /etc/gdnsd/geoip
Example gdnsd config for that setup.

plugins => { geoip => {
  maps => {
    prod => {
      geoip2_db => geo.mmdb,
      datacenters => [1,2,3],
      auto_dc_coords => {
       1 => [ 52.22, 21.01 ],
       2 => [ 1.35, 103.81 ],
       3 => [ 40.71, -74 ],
      }
    }
  },
  resources => {
    prod_www => {
      map => prod
      service_types => up
      dcmap => {
       1 => 1.1.1.1,
       2 => 2.2.2.2,
       3 => 3.3.3.3,
      }
    }
  }
}}

You can install gdnsd simply with apt-get.
gdnsd is not available for Ubuntu 22.04, however for 20.04, 23.04, Debian 10, Debian 11 and Debian 12.

Given the low memory usage and good performance by gdnsd, you can even run this with a fat .mmdb on low end boxes, like I do.

Enjoy.

You are receiving this email because your account has an Access Application with a wildcard definition that will begin to cover more URL combinations. We are updating our wildcard behavior in Cloudflare Access for wildcards at the end of a path not following a slash character (e.g. example.com/text*). If no action is taken before April 20th, 2023, an Access login screen will be presented for additional path combinations.

Current Access Application behavior
example.com/alpha* will cover example.com/alpha and example.com/alpha/one but not example.com/alphabet.

Change impact
After April 20th, 2023 at 20:00 UTC, all three path combinations will be covered by Access. If you would like to exempt specific paths from Access, a Bypass policy can be configured.

How to identify impacted Access Applications
To identify which Access Applications will be impacted by this change, please open the Zero Trust Dashboard, navigate to Access→Applications and search for the * character. This will highlight any applications that may require modification.

I consider this to be the logical way the wildcard should work - as it should have been from the start.

I've updated my Cloudflare Zero Trust article - as this wildcard function was one of my complaints.

Just wanted to get some input with my attempt and configuring native ISP ipv6 on my network. So for an overview of how things are setup, my ISP gives me a Cable Modem which has a static ipv4 and /64 ipv6 which feeds a pfsense and from there everything else. They do not allow me to put the modem in bridged mode.

lets use fd00:0824:2273:a::/64 as stand in for the public ipv6.

If I assign fd00:0824:2273:a::1 to the modem and fd00:0824:2273:a::2 to pfsense on WAN side. How can I go about make all clients on pfsense LAN side have an ipv6.

I have attempted using ULA on LAN side and then using NPt but it does not work. As soon as traffic gets to pfsense it stop. (Yes I have all appropriate pfsense rules and have verified in the logs that it is not simply being blocked)

I have attempted breaking down the /64 into 2 /65 and use 1 for wan and 1 for lan but no luck.

Is there a simpler way of doing this that I am just overlooking? I appreciate any help.

But more and more often lately, some IPs are blocked there and get http errors, so I bought a cheap domain and I woke up a $3/year @natvps_uk from idling, and..

wget -4 -qO- myip.cam
or for ipv6
wget -6 -qO- myip.cam

(or curl -4L myip.cam if curl is your thing)

https://github.com/imthenachoman/How-To-Secure-A-Linux-Server

With KVM and LXC and other virtualization technologies being very popular now, chroot might not seem needed very often any more. Nevertheless, I had a lot of fun last night making a chroot environment.

With debootstrap, the chroot was super fast and easy to set up. It seems to work okay, has network access, I can ssh in, etc.

Wikipedia's chroot article tells the very interesting history of chroot, beginning with its introduction in v7 Unix in 1979. The linked Wikipedia article also contains an interesting section on the limitations of chroot. However, the limitations might not be especially significant for every use case.

Is it too crazy to imagine that chroot still might be very useful today? For example, if all one needs is a quick test of something? As another example, Hetzner uses chroot in Installimage, which is one of Hetzner's methods to install Operating Systems on dedicated servers.

Here, from Installimage, are lines 9 through 27 of chroot.functions.sh, where execute_chroot_command() is defined:

execute_chroot_command_wo_debug() {
  local dirs=(/{dev,dev/pts,dev/shm,proc,run,sys})
  for dir in "${dirs[@]}"; do
    mkdir -p "$FOLD/hdd/$dir"
    mount --bind "$dir" "$FOLD/hdd/$dir"
  done
  unshare -f -p chroot "$FOLD/hdd" /usr/bin/env bash -c "$@"
  local r=$?
  for ((i=${#dirs[@]}-1; i>=0; i--)); do
    until umount "$FOLD/hdd/${dirs[i]}"; do :; done
  done
  return $r
}

execute_chroot_command() {
  debug "# chroot: $*"
  execute_chroot_command_wo_debug "$@" |& debugoutput
  return "${PIPESTATUS[0]}"
}

Please see also Installimage's functions.sh where execute_chroot_command() appears 14 times.

I hope to learn more about the unshare system call, which Installimage uses in the function execute_chroot_command_wo_debug(), and which "allows a process (or thread) to disassociate parts of its execution context that are currently being shared with other processes (or threads)."

People here at LES are invited to teach and learn together with me on MetalVPS. If you think you might like a MetalVPS account, please check the current offer at Free Debian Sid Intel i9-12900K Shell Accounts from MetalVPS!

I wonder how many people here still use chroot? And, besides Installimage, what are other current uses of chroot these days?

Friendly greetings!

Our server is an i9-9900K at Hetzner. Eight LXC containers are running various Linux distributions on Proxmox-VE 7.3. Everything seems to work great! . . . Except inside one Alpine Linux container. Somehow, IPv6 doesn't work from within this one container. Nevertheless, IPv6 ping to the container somehow seems to work from the Proxmox node. Amazingly, ping also works to the container even from the Wide Area Network (WAN).

How could such strangeness happen? Let's investigate!

A Closer Look At The Problem

The network configuration for all the containers was added individually by hand in the Proxmox GUI. Proxmox numbers LXC containers beginning with the number "100". The containers are IPv6 only. Each container was given a static IPv6 address ending with the container's Proxmox container number.

The container with the IPv6 network problem is number 102. An example of the working containers is number 106. Number 106 was chosen for comparison because both container 102 and container 106 were built from the same Alpine Linux LXC image. The other working containers are Ubuntu and Debian.

Alpine Linux seems less likely to be causing the IPv6 network failure because container 106, also built from the Alpine image, is functioning just fine. So let's look directly at the misbehaving container, number 102.

Container 102 can ping localhost.

root@Proxmox-VE ~ # lxc-attach -n 102
~ # ping6 -c 2 localhost
PING localhost (::1): 56 data bytes
64 bytes from ::1: seq=0 ttl=64 time=0.024 ms
64 bytes from ::1: seq=1 ttl=64 time=0.036 ms

--- localhost ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.024/0.030/0.036 ms
~ # 

Container 102 also responds to ping from external WAN.

[opc@instance-20220717-1620 ~]$ ping -c 2 2a01:4f8:121:24cc::102
PING 2a01:4f8:121:24cc::102(2a01:4f8:121:24cc::102) 56 data bytes
64 bytes from 2a01:4f8:121:24cc::102: icmp_seq=1 ttl=51 time=154 ms
64 bytes from 2a01:4f8:121:24cc::102: icmp_seq=2 ttl=51 time=154 ms

--- 2a01:4f8:121:24cc::102 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 153.966/154.042/154.118/0.076 ms
[opc@instance-20220717-1620 ~]$ 

However, from inside container 102, it can't ping its DNS servers, or the WAN.

root@Proxmox-VE ~ # lxc-attach -n 102
~ # cat /etc/resolv.conf
# --- BEGIN PVE ---
search metalvps.com
nameserver 2a01:4ff:ff00::add:2
nameserver 2001:470:20::2
# --- END PVE ---
~ # ping6 -c 2 2a01:4ff:ff00::add:2
PING 2a01:4ff:ff00::add:2 (2a01:4ff:ff00::add:2): 56 data bytes

--- 2a01:4ff:ff00::add:2 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
~ # ping6 -c 2 2001:470:20::2
PING 2001:470:20::2 (2001:470:20::2): 56 data bytes

--- 2001:470:20::2 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
~ # ping6 -c 2 metalvps.com
ping6: bad address 'metalvps.com'
~ # ping6 -c 2 ipv6.google.com
ping6: bad address 'ipv6.google.com'
~ # exit

By contrast, from inside working container 106, we can ping both the DNS servers and the WAN.

root@Proxmox-VE ~ # lxc-attach -n 106
~ # cat /etc/resolv.conf
# --- BEGIN PVE ---
search metalvps.com
nameserver 2a01:4ff:ff00::add:2
nameserver 2001:470:20::2
# --- END PVE ---
~ # ping6 -c 2 2a01:4ff:ff00::add:2
PING 2a01:4ff:ff00::add:2 (2a01:4ff:ff00::add:2): 56 data bytes
64 bytes from 2a01:4ff:ff00::add:2: seq=0 ttl=61 time=0.446 ms
64 bytes from 2a01:4ff:ff00::add:2: seq=1 ttl=61 time=0.274 ms

--- 2a01:4ff:ff00::add:2 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.274/0.360/0.446 ms
~ # ping6 -c 2 2001:470:20::2
PING 2001:470:20::2 (2001:470:20::2): 56 data bytes
64 bytes from 2001:470:20::2: seq=0 ttl=57 time=18.516 ms
64 bytes from 2001:470:20::2: seq=1 ttl=57 time=18.552 ms

--- 2001:470:20::2 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 18.516/18.534/18.552 ms
~ # ping6 -c 2 metalvps.com
PING metalvps.com (2603:c020:3:a9a9::250): 56 data bytes
64 bytes from 2603:c020:3:a9a9::250: seq=0 ttl=49 time=151.895 ms
64 bytes from 2603:c020:3:a9a9::250: seq=1 ttl=49 time=151.856 ms

--- metalvps.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 151.856/151.875/151.895 ms
~ # ping6 -c 2 ipv6.google.com
PING ipv6.google.com (2a00:1450:4001:830::200e): 56 data bytes
64 bytes from 2a00:1450:4001:830::200e: seq=0 ttl=118 time=5.477 ms
64 bytes from 2a00:1450:4001:830::200e: seq=1 ttl=118 time=5.611 ms

--- ipv6.google.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 5.477/5.544/5.611 ms
~ # 

Could The Firewall Cause This Problem?

The problem with container 102 probably is not a firewall drop egress issue because there aren't any egress rules in the container's zone and the default egress policy is ACCEPT. Also, the container's internal IPv6 egress problem still happens if container 102's firewall zone is set entirely off.

The ip Command

The ip command from the iproute2 suite provides much information about Linux networking. As the ip(8) man page explains, we can use ip in the following syntax: ip [OPTIONS] OBJECT { COMMAND | help }. In other words, to use ip we need to specify three items: an OPTION, an OBJECT, and a COMMAND.

For IPv4, the usual OPTION is nothing because ip defaults to IPv4. But, if we wish to specify the IPv4 OPTION explicitly, the option could be specified as "-family inet". The IPv4 OPTION can be abbreviated to "ip -f inet" or just "ip -4".

For IPv6, we use the OPTION "ip -family inet6". The IPv6 OPTION can be abbreviated to "ip -f inet6" or just "ip -6".

The OBJECT can be, for example, "link", "address", or "route". A "link" is a hardware or virtualized interface connecting our machine to a network. The "address" is the numerical IP address or addresses we assigned to our link. The "route" is the IP address where the kernel should send network packets initiated by programs running on our machine. "link", "address", and "route" can be abbreviated as "l", "a", and "r", respectively. Longer abbreviations also work. For example, "addr" commonly is used as an abbreviation for "address".

Frequently, when something happens involving networking, it can be very helpful to check the output of the ip command. Let's look with the ip command and see what we can find!

The Error Message

Checking the output of the ip -family inet6 address show command inside container 102 reveals "tentative dadfailed" as follows.

root@Proxmox-VE ~ # lxc-attach -n 102
~ # ip -f inet6 addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0@if45: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 state UP qlen 1000
    inet6 2a01:4f8:121:24cc::102/64 scope global tentative dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::d8dd:65ff:fe45:70f2/64 scope link 
       valid_lft forever preferred_lft forever
~ # exit
root@Proxmox-VE ~ # 

Output of the same command from inside container 106 where networking works fine does not include "tentative dadfailed."

root@Proxmox-VE ~ # lxc-attach -n 106
~ # ip -f inet6 addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0@if28: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 state UP qlen 1000
    inet6 2a01:4f8:121:24cc::106/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::64ce:78ff:fe85:15fd/64 scope link 
       valid_lft forever preferred_lft forever
~ # 

Dadfailed

Some quick googling found a page called IPv6 'dadfailed' Problem. The problem there involved reassigning an address previously assigned to another machine which failed without the address being deleted. However, down toward the bottom of the page, we find key information: the "dad" in "dadfailed" is "Duplicate Address Detection."

No Mistake With Container 102's Creation

But, even after reading about Duplicate Address Detection, I still did not immediately find the solution. I was puzzled about how, like the dadfailed page that Google showed me, some machine possibly could be down but with its address still present on my clean, new server with all its containers working fine except for this single itty bitty networking issue inside one container. Nothing was down. All the containers had their own addresses, and all the containers successfully started and continued running.

I kept looking and looking for some mistake I somehow might have made while setting up container 102. I checked and recheckd the screenshots that recorded container 102's creation. But everything looked okay!

Asking For Help

Next I talked with a well known hosting provider about what was happening. I also wrote up a LES post asking for help with the puzzling issue. It was starting to get late by the time I finished drafting the LES post. I decided to sleep. I deferred posting to LES Help until the next morning.

When I woke up, however, the idea was in my mind that possibly I mistakenly also had assigned the 102 address to another container. Such a mistake didn't seem likely, though, because there were only 8 containers, and the container numbers, 100, 101, 102, 103, 104, 105, 106, and 107 seemed understandable. I decided to check all the other containers besides 102 to make sure no additional container inadvertently and duplicatively also had been assigned the 102 address.

Checking All The Containers

• Container 100

First I checked container 100. Container 100 was the test container that had been checked by @yoursunny and found to be "very strong." Container 100 seemed to have been assigned the correct IP address ending in 100.

• Container 101

Uh! Oh! Here it is! Container 101 got the 102 address!!

How Did The Error Happen?

Container 101 was the second container that was created. Obviously it should have an IP address ending with 2, right? Well, not when the first address ended with 0.

What About The Node And WAN Responses To Ping?

How could container 102 respond to a ping from external WAN while seeming unable to initiate a ping to an external WAN destination? Container 101 already had been created with the 102 address. So, the ping responses came from container 101.

How Do We Fix container 102?

All that's needed to fix container 102 is to change its address to any that is not already in use. I tried the 101 address, and that address immediately worked fine.

More Information About Dadfailed

The ip command has a manual page which doesn't mention dadfailed. However, the ip man page refers to the ip-address man page, which does mention dadfailed as an addition to the ip address show command.

   ip address show - look at protocol addresses

        [ . . . ]

       dadfailed
              (IPv6 only) only list addresses which have failed duplicate ad‐
              dress detection.

       -dadfailed
              (IPv6 only) only list addresses which have not failed duplicate
              address detection.

For example, inside the Linux container on my Chromebook:

chronos@penguin:~/log$ ip -6 address show dadfailed
chronos@penguin:~/log$ ip -6 address show -dadfailed
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
5: eth0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::216:3eff:fe43:22fb/64 scope link 
       valid_lft forever preferred_lft forever
chronos@penguin:~/log$ 

Dadfailed In The RFCs

Duplicate Address Detection failure is discussed in section 5.4.5 of RFC 4862 which says:

5.4.5. When Duplicate Address Detection Fails

A tentative address that is determined to be a duplicate as described above MUST NOT be assigned to an interface, and the node SHOULD log a system management error.

Further discussion of Duplicate Address Detection occurs in Appendix A of RFC 4862 and in RFC 7527 Enhanced Duplicate Address Detection.

Dadfailed In The Logs

Indeed, the node did log a system management failure just as RFC 4862 said it should:

root@Proxmox-VE /var/log # zcat messages.2.gz | grep "duplicate address"
Jan 10 03:01:27 Proxmox-VE kernel: [97048.519242] IPv6: eth0: IPv6 duplicate address 2a01:4f8:121:24cc::102 used by 92:17:7a:22:c1:e2 detected!
[ . . . ]
Jan 13 06:44:36 Proxmox-VE kernel: [  141.636572] IPv6: eth0: IPv6 duplicate address 2a01:4f8:121:24cc::102 used by 92:17:7a:22:c1:e2 detected!
root@Proxmox-VE /var/log # 

Source Code

Recently I started to look at C source code. I don't know hardly anything about C, so looking at source code always is an adventure.

Above, we previously saw the ip command print the following "dadfailed" output:

~ # ip -f inet6 addr show
  [ . . . ]
2: eth0@if45: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 state UP qlen 1000
    inet6 2a01:4f8:121:24cc::102/64 scope global tentative dadfailed 
       valid_lft forever preferred_lft forever
  [ . . . ]

Can we find the source code which printed the dadfailed error? Finding the printing code sounds simple enough. Probably the printing code is nowhere near as complicated as the code which checked for and found the dadfailed error, right?

We saw the dadfailed error notification inside an Alpine container. Possibly that error notification came from iproute2 code running inside Busybox inside the Alpine container. Nevertheless, let's start by imagining how we might get the iproute2 source code that Proxmox itself uses. I'm unsure the method here gets the sources correctly, but we can try. First come the Proxmox sources, which seem to include the Debian sources, and then we get what might be the upstream sources.

root@Proxmox-VE ~ # git clone git://git.proxmox.com/git/iproute2.git
Cloning into 'iproute2'...
remote: Enumerating objects: 187, done.
remote: Total 187 (delta 0), reused 0 (delta 0), pack-reused 187
Receiving objects: 100% (187/187), 11.25 MiB | 88.63 MiB/s, done.
Resolving deltas: 100% (83/83), done.
root@Proxmox-VE ~ # 
cd iproute2/
root@Proxmox-VE ~/iproute2 # ls
debian  iproute2  Makefile  README
root@Proxmox-VE ~/iproute2 # cat README 
We compile our own package to make sure we always have the
latest version and bug fixes.
root@Proxmox-VE ~/iproute2 # ls debian/
changelog  copyright          iproute2-doc.examples  iproute2.links     README.Debian  source
compat     doc                iproute2-doc.install   iproute2.manpages  README.source
control    iproute2-doc.docs  iproute2.install       patches            rules
root@Proxmox-VE ~/iproute2 # ls iproute2/ #The iproute2/iproute2 directory is empty.
root@Proxmox-VE ~/iproute2 # git clone git://git.kernel.org/pub/scm/network/iproute2/iproute2.git
Cloning into 'iproute2'...
remote: Enumerating objects: 313, done.
remote: Counting objects: 100% (313/313), done.
remote: Compressing objects: 100% (289/289), done.
remote: Total 33973 (delta 60), reused 37 (delta 20), pack-reused 33660
Receiving objects: 100% (33973/33973), 8.34 MiB | 42.06 MiB/s, done.
Resolving deltas: 100% (25490/25490), done.
root@Proxmox-VE ~/iproute2 # 

Let's look inside the iproute2/iproute2 directory and then inside the iproute2/iproute2/ip directory.

root@Proxmox-VE ~/iproute2 # cd iproute2/
root@Proxmox-VE ~/iproute2/iproute2 # ls
bash-completion  COPYING  doc       genl     lib       misc   README        tc         vdpa
bridge           dcb      etc       include  Makefile  netem  README.devel  testsuite
configure        devlink  examples  ip       man       rdma   schema        tipc
root@Proxmox-VE ~/iproute2/iproute2 # cd ip
root@Proxmox-VE ~/iproute2/iproute2/ip # ls
ila_common.h           iplink.c            iplink_vlan.c    ipprefix.c          link_vti.c
ip6tunnel.c            iplink_can.c        iplink_vrf.c     iproute.c           link_xfrm.c
ipaddress.c            iplink_dsa.c        iplink_vxcan.c   iproute_lwtunnel.c  Makefile
ipaddrlabel.c          iplink_dummy.c      iplink_vxlan.c   iprule.c            nh_common.h
ip.c                   iplink_geneve.c     iplink_wwan.c    ipseg6.c            routel
ip_common.h            iplink_gtp.c        iplink_xdp.c     ipstats.c           rtm_map.c
ipfou.c                iplink_hsr.c        iplink_xstats.c  iptoken.c           rtmon.c
ipila.c                iplink_ifb.c        ipmacsec.c       iptunnel.c          static-syms.c
ipioam6.c              iplink_ipoib.c      ipmaddr.c        iptuntap.c          tcp_metrics.c
ipl2tp.c               iplink_ipvlan.c     ipmonitor.c      ipvrf.c             tunnel.c
iplink_amt.c           iplink_macvlan.c    ipmptcp.c        ipxfrm.c            tunnel.h
iplink_bareudp.c       iplink_netdevsim.c  ipmroute.c       link_gre6.c         xfrm.h
iplink_batadv.c        iplink_nlmon.c      ipneigh.c        link_gre.c          xfrm_monitor.c
iplink_bond.c          iplink_rmnet.c      ipnetconf.c      link_ip6tnl.c       xfrm_policy.c
iplink_bond_slave.c    iplink_team.c       ipnetns.c        link_iptnl.c        xfrm_state.c
iplink_bridge.c        iplink_vcan.c       ipnexthop.c      link_veth.c
iplink_bridge_slave.c  iplink_virt_wifi.c  ipntable.c       link_vti6.c
root@Proxmox-VE ~/iproute2/iproute2/ip # 

It was the address OBJECT which drew our interest, so let's look at ipaddress.c.

root@Proxmox-VE ~/iproute2/iproute2/ip # cat -n ipaddress.c | grep dadfailed
    68                  "           [-]tentative | [-]deprecated | [-]dadfailed | temporary |\n"
  1411          { .name = "dadfailed",          .mask = IFA_F_DADFAILED,        .readonly = true,      .v6only = true},
root@Proxmox-VE ~/iproute2/iproute2/ip # 

Line 68 seems to be part of the error message which is printed when ip doesn't understand the arguments given to it on the command line.

If we back up from line 1411 to line 1400, we can see the comment reproduced here, just below. Also, a little further down, at lines 1451 and 1452, there's code involving both "print" and "name." I'm guessing that maybe lines 1451 and 1452 might be responsible for printing "tentative" and "dadfailed."

  1400  /* Mapping from argument to address flag mask and attributes */
  1401  static const struct ifa_flag_data_t {
  [ . . . ]
  1411          { .name = "dadfailed",          .mask = IFA_F_DADFAILED,        .readonly = tru
e,       .v6only = true},
  [ . . . ]
  1414          { .name = "tentative",          .mask = IFA_F_TENTATIVE,        .readonly = tru
e,       .v6only = true},
  [ . . . ]
  1451                                  print_string(PRINT_FP, NULL,
  1452                                               "%s ", flag_data->name);

Finally, glancing at the Busybox ip command, the relevant source code seems to be ip.c. The usage message for ip address show is on lines 139 to 149.

139  //usage:#define ipaddr_trivial_usage
140  //usage:       "add|del IFADDR dev IFACE | show|flush [dev IFACE] [to PREFIX]"
141  //usage:#define ipaddr_full_usage "\n\n"
142  //usage:       "ipaddr add|change|replace|delete dev IFACE [CONFFLAG-LIST] IFADDR\n"
143  //usage:       "   IFADDR := PREFIX | ADDR peer PREFIX [broadcast ADDR|+|-]\n"
144  //usage:       "       [anycast ADDR] [label STRING] [scope SCOPE]\n"
145  //usage:       "   PREFIX := ADDR[/MASK]\n"
146  //usage:       "   SCOPE := [host|link|global|NUMBER]\n"
147  //usage:       "   CONFFLAG-LIST := [CONFFLAG-LIST] CONFFLAG\n"
148  //usage:       "   CONFFLAG := [noprefixroute]\n"
149  //usage:       "ipaddr show|flush [dev IFACE] [scope SCOPE] [to PREFIX] [label PATTERN]"

There is more on lines 341 to 347.

341  #if ENABLE_IPADDR
342  int ipaddr_main(int argc, char **argv) MAIN_EXTERNALLY_VISIBLE;
343  int ipaddr_main(int argc UNUSED_PARAM, char **argv)
344  {
345     return ip_do(do_ipaddr, argv);
346  }
347  #endif

ip_do is defined on lines 334 to 338.

334  static int ip_do(ip_func_ptr_t ip_func, char **argv)
335  {
336     argv = ip_parse_common_args(argv + 1);
337     return ip_func(argv);
338  }

Hopefully I soon will be able to understand more about how the iproute2 and Busybox code works to print the dadfailed error. Probably someone here at LES will be kind enough to provide some hints! Thanks very much! It is a lot of fun to learn a little about dadfailed!

Do you recommend using the PHP installer or Docker? I don't know that much about Docker, but shouldn't be that hard.

What's the best method to install, maintain and highly possible migrate later?

So, my two dedis are running manually set-up VMs, that I've set up using virsh and made my own script to create cloud-init configs, etc. Neither of these machines ever seem to have any problems with the VMs.

But when I upgraded my home network to use fibre, I decided to get one of those multi-port NUCs, and run pfSense (FreeBSD) virtualised on the free version of proxmox, which was set up with pretty much the default configuration, and following the guide from STH. This pfSense instance has been 100% reliable, never had any downtime with it.

But I also decided to make use of the spare capacity with a couple of smaller VMs, the main one being an extra borg backup destination for my most critical VMs from the dedis, but also a couple of ancillary ones that get very little use e.g. a proxy to a site that requires a whitelisted IP, so it uses wireguard to make a tunnel to one of my dedis, and an SSH jump host into my home network (that the borg clients use).

Now, all of these VMs have very little use, and so I don't often notice when they have issues (apart from borg which would trigger an email when the client fails to backup), so I don't know exactly when they fail, but sometimes after a couple of weeks of not really doing much, the CPU on these instances will suddenly jump to 100% and the machine won't respond to pings or input on the (serial) console. Interestingly, it's usually not borg that this happens to, but the really idle machines that are basically just forwarding things.

Anyway, when this happens, it seems like it's using 100% CPU, but it seems like the linux kernel itself isn't even running as it doesn't even respond to pings. The only solution is to connect to the proxmox interface, and force a stop/reset. The safer reboot/shutdown options are ignored, and because that just sends an ACPI event to the client.

Today, this just happened while I happened to the borg instance while I was able to jump on the proxmox interface to examine it, and it looks like the load suddenly jumped up while extra disk space was being allocated to the VM (proxmox seems to use these LVM sparse / lite volumes, which is the main difference between the instances on my dedis which are all allocated space using a pre-allocated volume group).

Now, the graphs only seem to have minute accuracy, but first jump in disk allocation corresponded to the start of the CPU spike from 0% to 50% (which is understandable as the borg server was actually doing something) and spike in network traffic, a minute later CPU hits 90% and traffic increases more, and another minute later, CPU is still 90%, traffic hits it maximum and the disk allocation increases again. The next minute, CPU is stuck at 100%, and network traffic is negligible (presumably ssh retrying to connect), and zero the minute after that.

So now my theory is that maybe this random proxmox thing is being triggered whenever a LVM lite allocation is increased, maybe because of logging, log rotation, or something else. Mostly, I'm concluding this as I've changed everything about the virtual hardware on these machines, so no matter what disk type, video card type, network card type, they all seem to end up like this stuck CPU state at some random point. Of course, it could just be a genuine hardware issue, but the fact that the pfsense instance has been rock solid (only rebooted once in the 7 months I've been using this setup, and that was because I moved the router to a new location and had to unplug it) makes me think there's some weirdness with proxmox itself...

There's no real reason for proxmox to be trying to use sparse LVM volumes anyway, as I've only allocated about half my 500G SSD to VMs, but it was the default setting when I set it up, so I left it alone.

Has anyone else seen weirdness like this in proxmox before? Or just as valid feedback, do most people just use the sparse LVM allocation and have no issues at all?

---------------------------------------------------------------------------   
OS           : CentOS 7.7.1908 (64 Bit)
 Virt/Kernel  : KVM / 5.4.5-1.el7.elrepo.x86_64
 CPU Model    : AMD Ryzen 7 3700X 8-Core Processor
 CPU Cores    : 2 @ 3593.248 MHz x86_64 512 KB Cache
 CPU Flags    : AES-NI Enabled & VM-x/AMD-V Disabled
 Load Average : 0.00, 0.00, 0.00
 Total Space  : 18G (13G ~67% used)
 Total RAM    : 1990 MB (679 MB + 912 MB Buff in use)
 Total SWAP   : 2047 MB (0 MB in use)
 Uptime       : 11 days 7:18
---------------------------------------------------------------------------
 ASN & ISP    : AS24940, Hetzner Online GmbH
 Organization :
 Location     : Falkenstein, Germany / DE
 Region       : Saxony
---------------------------------------------------------------------------

 ## Geekbench v4 CPU Benchmark:

  Single Core : 5170  (EXCELLENT)
   Multi Core : 9023

 ## IO Test

 CPU Speed:
    bzip2     : 136 MB/s
   sha256     :   1.4 GB/s
   md5sum     : 658 MB/s

 RAM Speed:
   Avg. write : 3072.0 MB/s
   Avg. read  : 8908.8 MB/s

 Disk Speed:
   1st run    : 1.7 GB/s
   2nd run    : 2.4 GB/s
   3rd run    : 2.4 GB/s
   -----------------------
   Average    : 2218.7 MB/s

 ## Global Speedtest

 Location                       Upload           Download         Ping
---------------------------------------------------------------------------
 Speedtest.net                  901.30 Mbit/s    878.05 Mbit/s    3.25 ms
 USA, New York (AT&T)           144.94 Mbit/s    125.09 Mbit/s   103.261 ms
 USA, Chicago (Windstream)      187.46 Mbit/s    110.34 Mbit/s   101.268 ms
 USA, Dallas (Frontier)         152.66 Mbit/s    168.53 Mbit/s   124.391 ms
 USA, Miami (Frontier)          168.47 Mbit/s    161.24 Mbit/s   113.602 ms
 USA, Los Angeles (Spectrum)    131.42 Mbit/s    148.38 Mbit/s   151.154 ms
 UK, London (Community Fibre)   580.25 Mbit/s    536.88 Mbit/s    19.524 ms
 France, Lyon (SFR)             510.47 Mbit/s    343.03 Mbit/s    19.664 ms
 Germany, Berlin (DNS:NET)      595.07 Mbit/s    649.75 Mbit/s    18.254 ms
 Spain, Madrid (MasMovil)       470.36 Mbit/s    508.52 Mbit/s    29.368 ms
 Italy, Rome (Unidata)          350.80 Mbit/s    363.33 Mbit/s    47.221 ms
 Israel, Haifa (013Netvision)   284.63 Mbit/s    292.39 Mbit/s    58.202 ms
 India, New Delhi (GIGATEL)     136.84 Mbit/s    285.17 Mbit/s   132.552 ms
 Singapore (FirstMedia)         39.11 Mbit/s     36.78 Mbit/s    170.950 ms
 Japan, Tsukuba (SoftEther)     18.34 Mbit/s     60.18 Mbit/s    250.445 ms
 Australia, Sydney (Yes Optus)  19.22 Mbit/s     16.35 Mbit/s    291.932 ms
 RSA, Randburg (Cool Ideas)     84.76 Mbit/s     219.46 Mbit/s   188.881 ms
 Brazil, Sao Paulo (Criare)     79.18 Mbit/s     206.16 Mbit/s   192.480 ms
---------------------------------------------------------------------------

 Finished in : 9 min 16 sec

https://vitobotta.com/2022/12/25/storing-grafana-loki-log-data-in-a-google-cloud-storage-bucket/

Happy holidays!

I've spend quite a lot of time setting up iGPU passthrough with an AMD processor. It was super easy with Intel's iGPU (VT-d) but not that easy with a Ryzen APU. Also thanks a lot for @VirtFusion for your efforts to implement such functionalities. In general this works quite similar with an external GPU or any other PCIe device, you just need to regroup the iommo-groups.

Guide

Enable IOMMU in Motherboard UEFI

As a first step, make sure that iommo is enabled in your motherboard's uefi. It should be really enabled and not in auto mode because I noticed that for some motherboards (e.g. MSI) auto doesn't work for some reason but enabled does.

Install OS and VirtFusion

Perform an installation from ISO without desktop environment (in theory you can use it along with a desktop programm but this will install packages which interfer with our setup). Install VirtFusion and configure networking and everything as described in their docs. Do not create a VM however. Also install pciutils which might not be part of a default installation but we'll need it later.

Server setup

Now comes the actual part of the passthrough procedure.
The hypervisor may not load any drivers that activate the iGPU because the iGPU can only be accessed by one driver per startup. We need to tell the hypervisor kernel to not load any video modules by appending the following to the line GRUB_CMDLINE_LINUX_DEFAULT=:

# nano /etc/default/grub

amd_iommu=on initcall_blacklist=sysfb_init video=simplefb:off video=vesafb:off video=efifb:off video=vesa:off disable_vga=1 textonly vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1 modprobe.blacklist=amdgpu,snd_hda_intel

Not all those flags might be necessary but they usually don't have any negative effects if they're set though they're not needed.
In a next step we need to load vfio drivers which are used as passthrough drivers that host the iGPU as device in the virtual machine later.

DEBIAN

 # nano /etc/modules

 vfio
 vfio_iommu_type1
 vfio_pci
 vfio_virqfd

RHEL

 # vi /etc/modules-load.d/vfio-pci.conf

 vfio
 vfio_iommu_type1
 vfio-pci
 vfio_virqfd

Then we need to tell vfio which devices it should pass through. You can look up the device IDs of the iommo-group through pciutils with lscpi -nnv. Note that you have to list the IDs of all devices of the iommo-group to be passed on even if you don't use the sound device for instance.

# nano /etc/modprobe.d/vfio.conf

options vfio-pci ids=1002:1863,1002:1864

Then we update the grup config to reflect the new settings and restart the server:

update-grub2 (RHEL: grub2-mkconfig -o /boot/grub2/grub.cfg)
update-initramfs -u -k all
reboot now

To verify you were successfull, run lspci -nnv and scroll down until you find your VGA compatible device and Audio device. If it says Kernel driver in use: vfio-pci you're all set.

VirtFusion setup

After having configured the hypervisor, it's time to create our first virtual machine in VirtFusion and configure it to use the iGPU through the previously created vfio virtual-devices.
It may work with seabios (VirtFusion: "SMBIOS") and i440FX (VirtFusion: "PC") but only on compatibility layer hence we use Q35 (which supports PCIe and not only PCI like i440FX) and UEFI (because the vbios, see below, will be available for UEFI only on modern systems).

The environmental variables of the virtual machine are now configured to work with vfio and an optional modern vbios. You can extract the vbios from the bios file of your motherboard vendor with e.g. GPU-Z. I've stored my vbios of the APU in this example at /usr/share/kvm/vbios-cezanne.bin.
In order to make libvirt create the hostdevice for us, we need to configure it in the corresponding fields in VirtFusion. For compatibility reasons you can hide the kvm status but this isn't mendatory. Make sure to adapt the iommu-group number and the functions id to the one of your config. You get the values with lscpi -nnv.

<hostdev mode="subsystem" type="pci" managed="yes">
    <driver name="vfio"/>
    <source>
        <address type="pci" domain="0x0000" bus="0x20" slot="0x00" function="0x0"/>
    </source>
    <address type="pci" domain="0x0000" bus="0x00" slot="0x8" function="0x0"/>
    <rom bar="on" file="/usr/share/kvm/vbios-cezanne.bin"/>
</hostdev>

Enable the custom elements you've filled out and update the config. Then restart the virtual machine.
Do not assign a PCIe device (in this case the iGPU) to more than one virtual machine at a time or you'll get permission errors from vfio or driver errors.
That's it. I hope this guide is helpful to you.

Now, I have been able to configure the cPanel module, and it works fine.
However, I'm not able to get the DirectAdmin module to load the created user packages (as shown in the attached screenshot).

I've tried disabling the SSL option in the module configuration - and that didn't help.

I have double checked the server hostname and login credentials - they work outside of Blesta with no problems.

PHP 7.4
Blesta: 5.5.3
Module: DirectAdmin 2.15.0

I suspect it's related to Blesta. Maybe it's just the interface. I've tried to get it to work now with MyW hosting, HostMantis, and with MXroute DA reseller.

Maybe DA did something during one of the updates. Maybe it's Blesta. Of course, it could always be some mistake on my part, but Google, Blesta, DirectAdmin (and MXroute for that matter) tutorials dosc and info aren't helping.

Here's a screenshot of the problem:

I have also tried removing the Group, as well as choosing the "Any" option for the group. No help.

Have searched online for tutorials but no luck finding something I can use. A common suggestion is just to set up a regular email server and block port for outgoing emails but am looking for the lightest weight solution.

Plan on using Thunderbird from home PC to view and archive and it's already set up with reputable third party for sending email. I should be able enable Dovecot, Postflix, Fetchmail, Qmail, Procmail Mail Filter, SpamAssasin Mail filter, and ClamAV with a couple mouse clicks in control panel, as needed.

If anyone is using a VPS for incoming mail only please let me know what worked best for you, what settings you use, and what I should avoid. If anyone knows of any good tutorials for doing this geared toward less experienced users would appreciate the info.

So the store would be mysite.com and blog would be blog.mysite.com and gallery would be gallery.mysite.com.

In a case like this is it best to use the same database for all three? And just use prefixes like bl_ and ga_ for the blog and gallery database tables? The database for the store currently does not use prefixes.

Any reason to use/not use subdomains?

Advice, tips, suggestions appreciated.

It runs properly only on Windows 10, and is a real hassle to configure.

I managed to make a VirtualBox machine that is installed, configured and ready to work (unactivated Win10, so anyone can activate it with their own Win key).

It's a 50 GB instance, that gets 7z-ed to a 13.5 GB file.

Now, how can I distribute it?
I wouldn't mind charging for it (I suppose any infrastructure for it would cost money, for bandwidth if nothing else).
But I wouldn't mind sharing it for free if there are no costs for me.

I expect a relatively steep rise of downloads, that will taper off to a low number of downloads over time.
A few thousands in total, I suppose (maybe more, maybe less).

Of course, it's possible that no one will download it, ever - lol.

history | grep searchTerm

ctrl+a and ctrl+e move to the beginning or end of the line.

also, installing fzf was a massive productivity booster while doing things like ctrl+r (https://www.youtube.com/watch?v=qgG5Jhi_Els)

Update:

I 'll use the comments below to write a summary article here, these might be more useful than random commands on the web as these are likely heavily used by the community here.

search through history:

history | grep searchTerm

Install fzf to introduce partial/fuzzy search for history, files etc.

ctrl+a and ctrl+e move to the beginning or end of the line.

ctrl+LEFT|RIGHT arrow to jump words

CTRL+L clear screen

TAB for autocomplete

fdupes -Sr . FDUPES is a program for identifying duplicate files residing
within specified directories.

jdupes a faster fdupes

ncdu to work out where all the storage space went

tldr <command> an easy to read manpage

last reboot Shows reboot history

findmnt -l list all mounts

iftop realtime bandwidth monitoring

iostat -m 10 report to better balance the input/output load between physical disks

ss -tupan socket statistics.

scp localFilePath user@remote:/remote/file/path remote file transfer.

nload - shows per interface network usage live on a graph on your terminal.

Cool Stuff:

telnet 2001:7b8:666:ffff::1:42

Won't tell you what it is, but it is safe.

ani-cli animename episode

watch anime on the terminal

apt-get install sl sl
cures bad habit of mistyping ls

Combinations:

zfs send -w tank/dana@snap1 | ssh host2 zfs recv newtank/dana

ZFS send and recv.

du -k ./* | sort -nr | cut -f2 | xargs -d '\n' du -sh | less

useful for debugging diskspace

ls -a; uptime; exit

ps -C sshd o user,pid,comm,lstart as alternative to who

On a tmux session, open multiple panes, then do this:
Ctrl+B, Shift+;

Then type: setw sync on All panes will now type the same thing

type setw sync off to turn it off again.

*sh aliases: https://tldp.org/LDP/abs/html/aliases.html

external Tools:
https://doublecmd.sourceforge.io/
https://github.com/junegunn/fzf
https://github.com/nvbn/thefuck - auto corrects last command
https://github.com/kellyjonbrazil/jc - terminal output as json
http://www.figlet.org/ - text to ascii
https://books.goalkicker.com/BashBook/ - there are other good books there

Contributors: @jperkins @typicalGta @xleet @ralf @CamoYoshi @Neoon @ehab @legendary @hornet @shallow @Ympker @Encoders @iansss @Mumbly @msatt @cold @Calin @g4m3r @havoc @vyas @TheDP

Thanks for all the contributions

Simply to say we should have four DNS records setup correctly to make the email hosting work as expected. Though this guide is using the mail hosting by ourselves, but the theory is almost the same for every provider. Thanks.

Greeting, Hope Everyone is safe and sound in good shape.
\

Well, Recently I installed a Adguard + Wireguard on a VPS Server. To my suprise am getting New Spam Clients from China and Some other Countries.

Especaillay pizzaseo .com& other clients.

So, kindly say how to stop these, and let me get to know that what i have messed up.

If it doesn't boots into software drivers (Compatibility/Advanced) I can't boot it.

Here is my paste.gg for the two times I tried booting it https://paste.gg/p/anonymous/8416287828884071877cb928529861b8

Is anybody able to help me troubleshoot this further on what this might be coming from??

• E5-2680 should have SGX under KVM
• EPYC 7543P seems like there is SEV support for KVM

Informally on a small number of hosts I haven't seen these enabled...
Is there a reason that these are usually disabled ?
Is it just too much of a niche feature for it to be set in Proxmox?

One way or another I ended up with a very reasonable (quite sure it's still single digits) number of VPN's. Most of them got more storage space than needed, but a couple could use some more space.

Is there a 'best' way to create some JBOD-like storage on one of my VPS out of left over bits of other VPS's?

I love SSHFS and use it to bolt all kinds of things together. It gives separate directories for each mount point, so with 5 VPS's that have 2 GB left, I can create 5 separate directories on the 6th server. Is there a way to homogenize those into a single 10 GB directory or volume?

I hardly have any experience with NFS or iSCSI, I think those won't be happy with the relatively high latency between hosts. Would an open source S3 storage framework work? Would Ceph or Gluster give what I look for?

The storage is mostly needed for Nextcloud. It doesn't need to be very fast. All VPS's run Debian.