@lesuser said:
It is reassuring to hear that there has been no update from VirMach after so many hours.
Was sleeping, I assumed due to the number of outages it was likely the monitoring service going down/having issues since it was across multiple datacenters/data carriers. Right now it looks like just Amsterdam but depending on where I look it varies. Doing a more extensive check now.
(edit) Okay more are appearing down again on monitoring randomly.
@VirMach it seems like everything 47.87.x.x is basically dead, with a few people/routes still working just barely (maybe old info left in those routers)
@Daevien said: @VirMach it seems like everything 47.87.x.x is basically dead, with a few people/routes still working just barely (maybe old info left in those routers)
Yeah, looks like it's the 47.87.128.0/17 block.
Theory #1: they changed their bank details and when sending the payment, without our knowledge it was delayed several days by our bank, it's still "pending review" by the bank. No notification from the bank or them, or any datacenter regarding a de-announcement request though and it still appears to be on our ARIN.
Could possibly also be unrelated and something else related to that. I'm contacting everyone now to see what's going on and also checking the routes to see how it's flapping. Contacting bank in the morning, already sent off something to the IP subnet provider. Reaching out/checking for communication from DCs now.
@Daevien said: @VirMach it seems like everything 47.87.x.x is basically dead, with a few people/routes still working just barely (maybe old info left in those routers)
Yeah, looks like it's the 47.87.128.0/17 block.
Theory #1: they changed their bank details and when sending the payment, without our knowledge it was delayed several days by our bank, it's still "pending review" by the bank. No notification from the bank or them, or any datacenter regarding a de-announcement request though and it still appears to be on our ARIN.
Could possibly also be unrelated and something else related to that. I'm contacting everyone now to see what's going on and also checking the routes to see how it's flapping. Contacting bank in the morning, already sent off something to the IP subnet provider. Reaching out/checking for communication from DCs now.
hello, man, u still alive?
i hope you are well and family is good.
you should be more active, at least in LES, we all have our pants down so you can feel at home
ehab
ps: edited for my finglish before @AlwaysSkint fingers me again.
I hope it can be resolved soon. Calling the bank as soon as they open or driving down there, still awaiting response by them, looks like records were modified and it's conflicting which explains how the issues transpired.
We're ready to do emergency switches if necessary, we can get blocks from IPXO, but best to avoid that as it may cause another ticket backlog. They're obviously building up already.
In any case it unfortunately does look like some issues will continue until everything updated back. It sure would be nice to own our own IPv4 blocks. I just need to take out a small loan of 2 million dollars for that. We might buy one or two /24 and sell it as an optional $50 one-time fee for a true static 1x IPv4 for those that need it to avoid future musical chairs with IPv4. All 2 customers that make that purchase will appreciate it.
https://billing.virmach.com/networkissuesrss.php had a change.
---
(changed) <![CDATA[VirMach® Network Issues Feed]]> https://billing.virmach.com/networkissues.php https://billing.virmach.com/networkissues.php?view=nid1488 Mon, 27 Mar 2023 07:39:00 -0700 <![CDATA[
(into ) <![CDATA[VirMach® Network Issues Feed]]> https://billing.virmach.com/networkissues.php https://billing.virmach.com/networkissues.php?view=nid1496 Wed, 05 Apr 2023 01:30:00 -0700 <![CDATA[
(into )
(into ) This appears to be a network outage. We'll update you with any details.
(into )
(into ) Update: this appears to be an issue with the IPv4 block 47.87.128.0/17
(into )
(into ) ]]> https://billing.virmach.com/networkissues.php?view=nid1488 Mon, 27 Mar 2023 07:39:00 -0700 <
Comments
LAXA024It is reassuring to hear that there has been no update from VirMach after so many hours.
Powerful Core i9 VPS (aff)| List of all VirMach VPS (aff)
yes.
no problem
nooo probblemo
Was sleeping, I assumed due to the number of outages it was likely the monitoring service going down/having issues since it was across multiple datacenters/data carriers. Right now it looks like just Amsterdam but depending on where I look it varies. Doing a more extensive check now.
(edit) Okay more are appearing down again on monitoring randomly.
@VirMach it seems like everything 47.87.x.x is basically dead, with a few people/routes still working just barely (maybe old info left in those routers)
Yeah, looks like it's the 47.87.128.0/17 block.
Theory #1: they changed their bank details and when sending the payment, without our knowledge it was delayed several days by our bank, it's still "pending review" by the bank. No notification from the bank or them, or any datacenter regarding a de-announcement request though and it still appears to be on our ARIN.
Could possibly also be unrelated and something else related to that. I'm contacting everyone now to see what's going on and also checking the routes to see how it's flapping. Contacting bank in the morning, already sent off something to the IP subnet provider. Reaching out/checking for communication from DCs now.
Thanks
hello, man, u still alive?
i hope you are well and family is good.
you should be more active, at least in LES, we all have our pants down so you can feel at home
ehab
ps: edited for my finglish before @AlwaysSkint fingers me again.
Always a pleasure to see you @netomx
LES • About • Donate • Rules • Support
I hope it can be resolved soon. Calling the bank as soon as they open or driving down there, still awaiting response by them, looks like records were modified and it's conflicting which explains how the issues transpired.
We're ready to do emergency switches if necessary, we can get blocks from IPXO, but best to avoid that as it may cause another ticket backlog. They're obviously building up already.
In any case it unfortunately does look like some issues will continue until everything updated back. It sure would be nice to own our own IPv4 blocks. I just need to take out a small loan of 2 million dollars for that. We might buy one or two /24 and sell it as an optional $50 one-time fee for a true static 1x IPv4 for those that need it to avoid future musical chairs with IPv4. All 2 customers that make that purchase will appreciate it.
Fucking Alibaba will make HetrixTools bankrupt with that kind of rate.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
virmach has just been alibaba-ed.
I bench YABS 24/7/365 unless it's a leap year.
Non Alibaba IP still alive and healthy btw.
https://microlxc.net/
I'm just very grateful that Tokyo isn't Alibaba'd or else we'd have 3,000 tickets right now.
You waited like a month to re-post this... and you using it wrong? Why would you like HT to bankrupt?!
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
One day Alibaba gives rDNS, next day Alibaba takes everything.
China! Blame China!
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
LOL
the best post i saw from you is the fish image.
when ever i see you post .. it itches not to repost it
sorry
will do again.
how are you now?
Wait, I can start ticket for each of my affected services and get SLA credits - grand total of 0.02$?!
Mmmmmm
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
Okey, the actual fuck is this. We sure this is Alibaba related - like if this is payment/bank issue it would be deleted from BGP so it should timeout from everything, not only part (?)
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
https://lowendspirit.com/uploads/editor/gi/ippw0lcmqowk.png
Welcome to the internet. This is one reason one of my friends constantly complains about BGP. It's why AWS can have widespread outages from one Amazon engineer fat-fingering something.
I was just typing that it'd be likely that other sites/connections would also "timeout", so I already took the affected VPS offline from my side earlier this morning to let the connections gradually move to other ones, and just when I'm typing this I get the message from 2 other monitoring points I have that it's not longer available. So that is confirmation.
Could come back pretty quickly when things are sorted out though...
Make that 3 by counting me in
Amsterdam and Frankfurt will receive IPv4 changes throughout the next several hours. Old IP will remain there most likely, plus new. We're sending out mass email now and then most likely you'll receive a ticket with the change. These will be blocks from Tokyo moving over so geolocation may be incorrect for some time and I don't know exact timeline of the change propagating everywhere.
I'm very very rusty on the scripts we use for these changes, so the level of notice we're able to provide depends on that, as the person usually handling these isn't here at 4AM.
Next up may end up being emergency migrations from HV over to nodes that have non-47.87.xx.xx blocks since we have to eventually do that anyway. I figure since it's going to be spotty/no connectivity for some time it won't hurt to do these now.
This means my previous message saying we're sending 10 to Ryzen plus 1 Epyc will just be cut down to 1 Epyc only, for some time, as I assume if we get the 47.87.xx.xx block back there will be a lot of complaints from people receiving IPv4 from that block in Tokyo.
Some hour ago died also my .de VPS from Tomazu/Webhosting24 and day isn't over yet
Have you noticed that TYO007 is offline? Eight hours have passed and no exceptions have been reported in the network status.
Any time someone reports a Tokyo outage, I am very skeptical until it is proven. Because I would accurately say 99% of the time it is an inaccurate report. However in this case I already saw that, and I'm aware. Our monitoring was showing it online but enough reports and also other report from 2nd monitoring system. I'm dealing with a larger emergency at the moment. The server is definitely facing an issue. It's partially online but effectively offline. I'll open a network status with more information as soon as possible.
Okay, thanks for your reply.
@VirMach unfortunately your billing website is dead now.
https://microlxc.net/