@AnthonySmith said: would be a good to have some community involvement in this.
Sounds like a plan. Willing to kick in a bit of cash too if a suitably safe method can be found. (via inception preferably)
I'd personally be super interested in having some of the bigger players listed as reference too. I know LES intuitively is better bang per buck, but can't really visualize how it compares to say vultr and say GCP.
Also think it would make sense to work out how to aggregate this meaningfully before everyone goes missioning off.
This contains the sort of benchmarks I was referring to, I tried to get a real world view of what performance I could actually expect rather than a point in time snapshot of performance.
This contains the sort of benchmarks I was referring to, I tried to get a real world view of what performance I could actually expect rather than a point in time snapshot of performance.
This contains the sort of benchmarks I was referring to, I tried to get a real world view of what performance I could actually expect rather than a point in time snapshot of performance.
This contains the sort of benchmarks I was referring to, I tried to get a real world view of what performance I could actually expect rather than a point in time snapshot of performance.
Good write up, Ant!
Just a quick thought I had -- for your extended fio tests, you're limiting the block size to just 64k, when in reality the block size will likely be higher as the system tries to use what is most optimal. The yabs test shows the disk performs very well for the higher block sizes (512k or 1m), which is also using fio for non-sequential, random mixed read and writes. So I'm not entirely sure the bs=64k fio tests are really representative of system performance.
@AnthonySmith said: @Mason ok so upon doing some reading I should probably pick 4k or 512k going forward, I will probably go with 4k unless anyone thinks different?
If given the option, I'd opt for 512k.
If you look through the YABS results thread, nearly every single test does better (in bandwidth terms) with bs=512k over bs=4k. The low block sizes are really just to stress test how many IOPS you can get out of the machine. From the light reading I did when researching this problem many months ago, the disk controller and caching is going to auto-optimize the block size anyway and adjust as necessary to get the best throughput, with the direct=1 flag, fio is bypassing these mechanisms so setting the block size that low (4k) seems not too realistic (at least to me. open for debate).
And that's essentially the reasoning behind why I put multiple fio tests in the yabs script with increasing block sizes, so that the trade off between raw bandwidth and IOPS can be seen.
@Mason said: nearly every single test does better (in bandwidth terms) with bs=512k over bs=4k.
I am thinking in terms of real-world uncached random file access and database reads, which is going to account for a huge majority of real-world use cases, one of the studies I read over was by EMC (SAN makers) and the report concluded that the difference between 4k and 512k was essentially just a margin of error and then I made the assumption that 4k is more likely to be aligned for simulating actual use in most cases.
It's an interesting one, I honestly don't feel qualified to say for sure 1 way or the other but I will keep researching, I wonder if I can somehow get stats from a number of VPS nodes to see what is most aligned 4,64,512k or 1m
@AnthonySmith said: I am thinking in terms of real-world uncached random file access and database reads, which is going to account for a huge majority of real-world use cases, one of the studies I read over was by EMC (SAN makers) and the report concluded that the difference between 4k and 512k was essentially just a margin of error and then I made the assumption that 4k is more likely to be aligned for simulating actual use in most cases.
It's an interesting one, I honestly don't feel qualified to say for sure 1 way or the other but I will keep researching, I wonder if I can somehow get stats from a number of VPS nodes to see what is most aligned 4,64,512k or 1m
Could be the case! It probably all boils down to what is actually being represented in the test. Are we copying a large video file or making many database updates to many distinct tables, rows? I'd imagine the fio parameters would vary widely in trying to represent those two cases. (just some food for thought )
Edit:
And... just so I can pick on you and play devil's advocate...
report concluded that the difference between 4k and 512k was essentially just a margin of error
If that was the case, I'm shocked the results vary so widely. You're getting way different throughput depending on which block size you go with.
more likely to be aligned for simulating actual use in most cases
If we're simulating real use case, you'd probably want to use a mixed R/W test rather than a pure read or pure write test. yabs uses a 50/50 split, but real-world is probably more like 60(R)/40(W) or 65(R)/35(W). Again, just more food for thought
@Mason said: If we're simulating real use case, you'd probably want to use a mixed R/W test rather than a pure read or pure write test. yabs uses a 50/50 split, but real-world is probably more like 60(R)/40(W) or 65(R)/35(W). Again, just more food for thought
makes total sense, I think given that, perhaps the best thing to do is rather than a separate read and write test I do a 50/50 split as you suggested and do 1 at 4k and 1 at 512k and graph them both.
@Mason said: If we're simulating real use case, you'd probably want to use a mixed R/W test rather than a pure read or pure write test. yabs uses a 50/50 split, but real-world is probably more like 60(R)/40(W) or 65(R)/35(W). Again, just more food for thought
makes total sense, I think given that, perhaps the best thing to do is rather than a separate read and write test I do a 50/50 split as you suggested and do 1 at 4k and 1 at 512k and graph them both.
Sounds like a good idea to me! Would make some interesting graphs if you plot the read, write, and cumulative speed results independent of each other on the same graph.
@Mason said: If we're simulating real use case, you'd probably want to use a mixed R/W test rather than a pure read or pure write test. yabs uses a 50/50 split, but real-world is probably more like 60(R)/40(W) or 65(R)/35(W). Again, just more food for thought
makes total sense, I think given that, perhaps the best thing to do is rather than a separate read and write test I do a 50/50 split as you suggested and do 1 at 4k and 1 at 512k and graph them both.
Sounds like a good idea to me! Would make some interesting graphs if you plot the read, write, and cumulative speed results independent of each other on the same graph.
In the end I went with 4,64,512 and 1m will be an interesting graph I have just set off a 24-hour net and disk test on 2 servers at 11:00 am (UK/London) I will stop it just after 11:00 am tomorrow and graph the results for a new post.
Comments
Sounds like a plan. Willing to kick in a bit of cash too if a suitably safe method can be found. (via inception preferably)
I'd personally be super interested in having some of the bigger players listed as reference too. I know LES intuitively is better bang per buck, but can't really visualize how it compares to say vultr and say GCP.
Also think it would make sense to work out how to aggregate this meaningfully before everyone goes missioning off.
I have a parser for a few popular benchmarks that cleans up the results into a Json file for easier viewing/inserting into DB
https://lowendspirit.com/pq-hosting-moldova-netherlands-russia-latvia-ukraine-and-hong-kong-unmetered-bw
This contains the sort of benchmarks I was referring to, I tried to get a real world view of what performance I could actually expect rather than a point in time snapshot of performance.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
What a good find. "Hong King" for EUR 3,00
I bench YABS 24/7/365 unless it's a leap year.
As @SmallWeb says...
Noice!
A bit of spellchecking (comments in writers corner) and we should be off to the races.
VPS reviews and benchmarks |
Good write up, Ant!
Just a quick thought I had -- for your extended fio tests, you're limiting the block size to just 64k, when in reality the block size will likely be higher as the system tries to use what is most optimal. The yabs test shows the disk performs very well for the higher block sizes (512k or 1m), which is also using fio for non-sequential, random mixed read and writes. So I'm not entirely sure the bs=64k fio tests are really representative of system performance.
Head Janitor @ LES • About • Rules • Support • Donate
I felt 64k would be more representative of real world workload, but now I type that, I have no rational behind it so I will do some further reading.
I am not looking for maximum performance, I am looking for real work load performance or as close as I can get when benchmarking.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
I would like to say hong king is a joke based on British ownership, but it was just a typo
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
Hong Kong
King Kong
-FTFY
VPS reviews and benchmarks |
@Mason ok so upon doing some reading I should probably pick 4k or 512k going forward, I will probably go with 4k unless anyone thinks different?
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
Even better, There is one for 1€ (stocked out).
Someone should make a scrapper for this one
https://phpbackend.com/
If given the option, I'd opt for 512k.
If you look through the YABS results thread, nearly every single test does better (in bandwidth terms) with bs=512k over bs=4k. The low block sizes are really just to stress test how many IOPS you can get out of the machine. From the light reading I did when researching this problem many months ago, the disk controller and caching is going to auto-optimize the block size anyway and adjust as necessary to get the best throughput, with the direct=1 flag, fio is bypassing these mechanisms so setting the block size that low (4k) seems not too realistic (at least to me. open for debate).
And that's essentially the reasoning behind why I put multiple fio tests in the yabs script with increasing block sizes, so that the trade off between raw bandwidth and IOPS can be seen.
Head Janitor @ LES • About • Rules • Support • Donate
I am thinking in terms of real-world uncached random file access and database reads, which is going to account for a huge majority of real-world use cases, one of the studies I read over was by EMC (SAN makers) and the report concluded that the difference between 4k and 512k was essentially just a margin of error and then I made the assumption that 4k is more likely to be aligned for simulating actual use in most cases.
It's an interesting one, I honestly don't feel qualified to say for sure 1 way or the other but I will keep researching, I wonder if I can somehow get stats from a number of VPS nodes to see what is most aligned 4,64,512k or 1m
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
Could be the case! It probably all boils down to what is actually being represented in the test. Are we copying a large video file or making many database updates to many distinct tables, rows? I'd imagine the fio parameters would vary widely in trying to represent those two cases. (just some food for thought
)
Edit:
And... just so I can pick on you and play devil's advocate...
If that was the case, I'm shocked the results vary so widely. You're getting way different throughput depending on which block size you go with.
If we're simulating real use case, you'd probably want to use a mixed R/W test rather than a pure read or pure write test. yabs uses a 50/50 split, but real-world is probably more like 60(R)/40(W) or 65(R)/35(W). Again, just more food for thought
Head Janitor @ LES • About • Rules • Support • Donate
makes total sense, I think given that, perhaps the best thing to do is rather than a separate read and write test I do a 50/50 split as you suggested and do 1 at 4k and 1 at 512k and graph them both.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
Sounds like a good idea to me! Would make some interesting graphs if you plot the read, write, and cumulative speed results independent of each other on the same graph.
Head Janitor @ LES • About • Rules • Support • Donate
In the end I went with 4,64,512 and 1m will be an interesting graph
I have just set off a 24-hour net and disk test on 2 servers at 11:00 am (UK/London) I will stop it just after 11:00 am tomorrow and graph the results for a new post.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.