the thing is that rwmix=50 write IOPS reflected are high.
at 75 the read is higher, write is lower, which does not reflect what the highest IOPS are.
i dont know maybe do a dedicated half file size of read, and then half write (e.g. 2GB read test, 2GB write test) maybe, or if rwmix does both operations at the same time, then maybe it does simulate real world performance, then in any case it would be nice if there's a reference grading based on IOPS results from:
LEB Summerhost
Bad
Mediocre
Good
Excellent
NVMe
in the context of rwmix=75 assuming this is close to real usage.
also when doing fio sometime vps i observe IOPS drop to 0 (fluctuate up and down), but the result end up to be a good average.
if this behaviour is concluded to be bad for real usage, maybe it can be discussed for future tests to also detect this "anomaly"
@poisson said: I think it is better to read and write the same amounts of data.
You're very unlikely to hit that in real life though - Usually you have far more reads vs writes, except in some particular cases (eg. a database server could be write-heavy if it's primarily for transactions instead of analytics).
I don't disagree, but for me when it is 50-50 with a 4GB test file size, I think there is enough data both reading and writing to give a good overall view of performance. I would also argue that with a 50-50 yes, the reading IOPS will go down, but you are likely to get performance that exceeds this value, while for writing, you get a value that is closer to the true capability of the disk than one that is artificially low and may make people mistakenly conclude the writing sucks (and I think a lot of people want to see accurate writing speeds more than they want to see flashy read numbers).
I don't think we can ever come to an agreement what's the best, but I think 50-50 is good for the aforementioned reasons.
We'll probably all have to agree to disagree. The past few comments really showcase something that I've been struggling with -- creating a "one-size-fits-all" test is HARD. lol
hey @Mason thanks a lot for your work! as you probably already know I love it!
now finally everything I like to use for testing (geekbench, fio, iperf) is bundled in one script and don't need to lookup download-links and stuff anymore.
Big update just pushed to the repo finally with a whole lot of goodies (@Falzo - fio is finally here!). I tested the script with pretty much every Hetzner and Vultr CentOS, Ubuntu, Debian, and Fedora templates to make sure it's pretty robust and compatible with most Linux-based OSes..
Biggest updates:
fio - fio disk benchmarking tool is added to the script in place of the sequential (dd) test
IPv6-only compatibility - the script will now work on IPv6-only VMs (instructions in readme) (Geekbench 4 won't work since it requires IPv4 to download the files - might look into workaround)
32-bit compatibility - if a 32-bit architecture is detected, then the 32-bit fio and iperf3 binaries will be downloaded instead
Pre-compiled binaries are included in the project's bin directory. The readme explains why these are necessary, but it pretty much boils down to attempting to make running the script as simple as possible for the user. Please test it out and let me know if you have any trouble with the new script. I've tried to make it as robust as possible, but I'm sure I've missed some edge cases
@Mason said:
Big update just pushed to the repo finally with a whole lot of goodies (@Falzo - fio is finally here!). I tested the script with pretty much every Hetzner and Vultr CentOS, Ubuntu, Debian, and Fedora templates to make sure it's pretty robust and compatible with most Linux-based OSes..
Biggest updates:
fio - fio disk benchmarking tool is added to the script in place of the sequential (dd) test
IPv6-only compatibility - the script will now work on IPv6-only VMs (instructions in readme) (Geekbench 4 won't work since it requires IPv4 to download the files - might look into workaround)
32-bit compatibility - if a 32-bit architecture is detected, then the 32-bit fio and iperf3 binaries will be downloaded instead
Pre-compiled binaries are included in the project's bin directory. The readme explains why these are necessary, but it pretty much boils down to attempting to make running the script as simple as possible for the user. Please test it out and let me know if you have any trouble with the new script. I've tried to make it as robust as possible, but I'm sure I've missed some edge cases
@CamoYoshi said:
Hey @Mason... yabs.sh is available to buy right now... maybe we can have a shortened URL? Pretty please?
I was eyeballing that domain a couple weeks ago. Screw it... let's do it. Just registered it, I'll set up a shortened URL soon(tm). I'll probably just have it redirect to the github page fwiw. I'd rather keep all files and such on there for openness sake and so I don't have to worry about syncing the files.
@CamoYoshi said:
Hey @Mason... yabs.sh is available to buy right now... maybe we can have a shortened URL? Pretty please?
I was eyeballing that domain a couple weeks ago. Screw it... let's do it. Just registered it, I'll set up a shortened URL soon(tm). I'll probably just have it redirect to the github page fwiw. I'd rather keep all files and such on there for openness sake and so I don't have to worry about syncing the files.
Could also set up yabs.sh/run or something as a 301 redirect to the raw.githubblahblah URL too -- and perhaps a yabs.sh/run6 for the IPv6 version!
Thank you, BTW! I will have to try this on my servers this weekend.
@poisson said:
Nice going. Waiting for the update to Geekbench 5 next!
That's the next change I have planned right now. I'll let users pass a flag to still use the Geekbench 4 test, but once I have Geekbench 5 test in there, that'll be the new default.
@CamoYoshi said:
Hey @Mason... yabs.sh is available to buy right now... maybe we can have a shortened URL? Pretty please?
I was eyeballing that domain a couple weeks ago. Screw it... let's do it. Just registered it, I'll set up a shortened URL soon(tm). I'll probably just have it redirect to the github page fwiw. I'd rather keep all files and such on there for openness sake and so I don't have to worry about syncing the files.
Could also set up yabs.sh/run or something as a 301 redirect to the raw.githubblahblah URL too -- and perhaps a yabs.sh/run6 for the IPv6 version!
Thank you, BTW! I will have to try this on my servers this weekend.
Domain is set up now with appropriate redirects.
curl -sL yabs.sh | bash
How's that for short?
Redirection is based on user agent. Curl/wget requests will be redirected to the raw script content, while http requests will go to the main github project page. Thanks again for the neat idea! I'll get the docs updated soon.
Edit: Will have to test and work on IPv6 as I don't think this will work yet on IPv6-only VMs.
@CamoYoshi said:
Hey @Mason... yabs.sh is available to buy right now... maybe we can have a shortened URL? Pretty please?
I was eyeballing that domain a couple weeks ago. Screw it... let's do it. Just registered it, I'll set up a shortened URL soon(tm). I'll probably just have it redirect to the github page fwiw. I'd rather keep all files and such on there for openness sake and so I don't have to worry about syncing the files.
Could also set up yabs.sh/run or something as a 301 redirect to the raw.githubblahblah URL too -- and perhaps a yabs.sh/run6 for the IPv6 version!
Thank you, BTW! I will have to try this on my servers this weekend.
Domain is set up now with appropriate redirects.
curl -sL yabs.sh | bash
How's that for short?
Redirection is based on user agent. Curl/wget requests will be redirected to the raw script content, while http requests will go to the main github project page. Thanks again for the neat idea! I'll get the docs updated soon.
Edit: Will have to test and work on IPv6 as I don't think this will work yet on IPv6-only VMs.
You sir are the MVP of benchmarking scripts IMO. Thank you!!!
Geekbench 5 - using v5 test by default now (can be overridden with the '-4' flag)
Local fio/iperf3 Packages - local packages will take precedence over precompiled binaries, so if fio and/or iperf3 are already installed on the tested system, it will use the local packages instead. Should help iron out the incompatibility issues with the precompiled binaries
dd Fallback - if the fio disk test fails for whatever reason, the script will run a dd test instead and show those results so the disk is tested in some way rather than not at all
Added a 'reduce' option ('-r' flag), which will reduce the number of tested iperf3 locations to reduce the bandwidth burden on the tested system. It will limit the iperf3 tests to only use three locations (Online.net (FR), WorldStream (NL), and Hurricane Electric (US))
Added a help printout option ('-h' flag), which can be treated as a dry-run for the script. It will tell you the usage, accepted flags, detected host arch (x64/x86), detected flags (whatever you passed it along with the -h flag), and the local package check (if preinstalled fio/iperf3 are detected)
Please report any errors to me and I'll get them addressed. Cheers!
Here's the sample help output --
~/yet-another-bench-script$ ./yabs.sh -id4hr
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
# Yet-Another-Bench-Script #
# v2020-02-10 #
# https://github.com/masonr/yet-another-bench-script #
# ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## #
Mon Feb 10 23:50:10 EST 2020
Usage: ./yabs.sh [-fdighr4]
curl -sL yabs.sh | bash
curl -sL yabs.sh | bash -s -- -{fdighr4}
Flags:
-f/d : skips the fio disk benchmark test
-i : skips the iperf network test
-g : skips the geekbench performance test
-h : prints this lovely message, shows any flags you passed,
shows if fio/iperf3 local packages have been detected,
then exits
-r : reduce number of iperf3 network locations (to only three)
to lessen bandwidth usage
-4 : use geekbench 4 instead of geekbench 5
Detected Arch: x64
Detected Flags:
-f/d, skipping fio disk benchmark test
-i, skipping iperf network test
-r, using reduced (3) iperf3 locations
-4, running geekbench 4 instead of 5
Local Binary Check:
fio not detected, will download precompiled binary
iperf3 detected, using local package
Exiting...
Just tested with a few flags on different setups, work great.
The reason I brought imthe bandwidth issue was - because I have a super cheap plan ($4/year) that comes with 100 gb/year. I ran two yabs instances - once because session cut mid way . Fortunately realized the bandwidth impact and let the vps idle for the remainder of the billing period.
Sort of.
Comments
the thing is that rwmix=50 write IOPS reflected are high.
at 75 the read is higher, write is lower, which does not reflect what the highest IOPS are.
i dont know maybe do a dedicated half file size of read, and then half write (e.g. 2GB read test, 2GB write test) maybe, or if rwmix does both operations at the same time, then maybe it does simulate real world performance, then in any case it would be nice if there's a reference grading based on IOPS results from:
in the context of rwmix=75 assuming this is close to real usage.
also when doing fio sometime vps i observe IOPS drop to 0 (fluctuate up and down), but the result end up to be a good average.
if this behaviour is concluded to be bad for real usage, maybe it can be discussed for future tests to also detect this "anomaly"
I bench YABS 24/7/365 unless it's a leap year.
I don't disagree, but for me when it is 50-50 with a 4GB test file size, I think there is enough data both reading and writing to give a good overall view of performance. I would also argue that with a 50-50 yes, the reading IOPS will go down, but you are likely to get performance that exceeds this value, while for writing, you get a value that is closer to the true capability of the disk than one that is artificially low and may make people mistakenly conclude the writing sucks (and I think a lot of people want to see accurate writing speeds more than they want to see flashy read numbers).
I don't think we can ever come to an agreement what's the best, but I think 50-50 is good for the aforementioned reasons.
Deals and Reviews: LowEndBoxes Review | Avoid dodgy providers with The LEBRE Whitelist | Free hosting (with conditions): Evolution-Host, NanoKVM, FreeMach, ServedEZ | Get expert copyediting and copywriting help at The Write Flow
We'll probably all have to agree to disagree. The past few comments really showcase something that I've been struggling with -- creating a "one-size-fits-all" test is HARD. lol
Head Janitor @ LES • About • Rules • Support • Donate
hey @Mason thanks a lot for your work! as you probably already know I love it!
now finally everything I like to use for testing (geekbench, fio, iperf) is bundled in one script and don't need to lookup download-links and stuff anymore.
finally signed up here just to let you know ;-)
Big update just pushed to the repo finally with a whole lot of goodies (@Falzo - fio is finally here!). I tested the script with pretty much every Hetzner and Vultr CentOS, Ubuntu, Debian, and Fedora templates to make sure it's pretty robust and compatible with most Linux-based OSes..
Biggest updates:
Pre-compiled binaries are included in the project's bin directory. The readme explains why these are necessary, but it pretty much boils down to attempting to make running the script as simple as possible for the user. Please test it out and let me know if you have any trouble with the new script. I've tried to make it as robust as possible, but I'm sure I've missed some edge cases
Here's the current output --
Head Janitor @ LES • About • Rules • Support • Donate
You are not sexy.
You are SEX!
Nexus Bytes Ryzen Powered NVMe VPS | NYC|Miami|LA|London|Netherlands| Singapore|Tokyo
Storage VPS | LiteSpeed Powered Web Hosting + SSH access | Switcher Special |
Hey @Mason... yabs.sh is available to buy right now... maybe we can have a shortened URL?
Pretty please?
Cheap dedis are my drug, and I'm too far gone to turn back.
I was eyeballing that domain a couple weeks ago. Screw it... let's do it. Just registered it, I'll set up a shortened URL soon(tm). I'll probably just have it redirect to the github page fwiw. I'd rather keep all files and such on there for openness sake and so I don't have to worry about syncing the files.
Head Janitor @ LES • About • Rules • Support • Donate
Awesome updates to my favorite bench script. Thank you @Mason!
It don’t be like it is until it do.
Could also set up yabs.sh/run or something as a 301 redirect to the raw.githubblahblah URL too
-- and perhaps a yabs.sh/run6 for the IPv6 version!
Thank you, BTW! I will have to try this on my servers this weekend.
Cheap dedis are my drug, and I'm too far gone to turn back.
Nice going. Waiting for the update to Geekbench 5 next!
Deals and Reviews: LowEndBoxes Review | Avoid dodgy providers with The LEBRE Whitelist | Free hosting (with conditions): Evolution-Host, NanoKVM, FreeMach, ServedEZ | Get expert copyediting and copywriting help at The Write Flow
That's the next change I have planned right now. I'll let users pass a flag to still use the Geekbench 4 test, but once I have Geekbench 5 test in there, that'll be the new default.
Head Janitor @ LES • About • Rules • Support • Donate
Domain is set up now with appropriate redirects.
curl -sL yabs.sh | bashHow's that for short?
Redirection is based on user agent. Curl/wget requests will be redirected to the raw script content, while http requests will go to the main github project page. Thanks again for the neat idea! I'll get the docs updated soon.
Edit: Will have to test and work on IPv6 as I don't think this will work yet on IPv6-only VMs.
Head Janitor @ LES • About • Rules • Support • Donate
You sir are the MVP of benchmarking scripts IMO. Thank you!!!
Cheap dedis are my drug, and I'm too far gone to turn back.
Thanks @Mason for the tweaks and folks for suggestions ....
One silly question: is it typical to see @ 20-24 GB bandwidth consumption for every instance the updated YABS is run?
Thanks
VPS reviews and benchmarks |
Honestly, I'm not sure but that's something I should probably test and put in the readme somewhere
Head Janitor @ LES • About • Rules • Support • Donate
Nah, it usually uses 40,503GB. Ya know, and at least 100% of each core for 10 days. Gotta make sure that $2/mo service you bought is prem.
My pronouns are asshole/asshole/asshole. I will give you the same courtesy.
Actually for a $4 /year, 2 00 GB/month plan I will only be able to run YABS less than 8 times. That’s the concern. Got to keep that idler in shape.
VPS reviews and benchmarks |
You can always SIGKILL/SIGSTOP it and hope it doesn't use any real ticks.. might be able to double that.
My pronouns are asshole/asshole/asshole. I will give you the same courtesy.
looks good .
Thanks @Mason I'm going to add it to my Knowledgebase for my clients to use, as it will be great for them to have!
█ VPS Aliens | AS207328 | Your Portal to Alien Class Servers & Support!
█ VPS Solutions | Dedicated Servers | KVM VPS Reseller | 10Gbps Ports
Newest batch of updates were just pushed --
Please report any errors to me and I'll get them addressed. Cheers!
Here's the sample help output --
Head Janitor @ LES • About • Rules • Support • Donate
iperf can use a lot of data, and you need enough data transfer to get a more accurate result
Deals and Reviews: LowEndBoxes Review | Avoid dodgy providers with The LEBRE Whitelist | Free hosting (with conditions): Evolution-Host, NanoKVM, FreeMach, ServedEZ | Get expert copyediting and copywriting help at The Write Flow
@Mason
Thanks for the updates. The-r flag helps.
Just tested with a few flags on different setups, work great.
The reason I brought imthe bandwidth issue was - because I have a super cheap plan ($4/year) that comes with 100 gb/year. I ran two yabs instances - once because session cut mid way . Fortunately realized the bandwidth impact and let the vps idle for the remainder of the billing period.
Sort of.
@poisson,
I am aware of the impact of iperf- albeit only recent months
My concern was regarding plans with limited bandwidth as mentioned above.
Add to that the potential for overage charges or crippling speeds or disabling connectivity as per the TOS of the network provider .
Unfortunately things the way they are, one might end up seeing
“hey running your tests caused me overage charges....”
Kind of complaints instead of
“Thank you”
We saw this week how someone complained about a ‘free’ hosting service not working.
Alas....
(End preaching)
VPS reviews and benchmarks |
Idk what causes it..
Not enough RAM and no swap.
Geekbench 4 works perfectly
Well, as that's an older version it's probably expected. I've had trouble running Geekbench 5 on <2GB ram machines too.
Yep, probably what @debaser said.
Did you try running it manually (https://www.geekbench.com/)? What errors do you get?
Head Janitor @ LES • About • Rules • Support • Donate
Adding swap was nescessary for GeekBench5 to work on my 1GB VPS.