FlowVPS Message- Data array corruption

vyasvyas OGRetired
edited August 2020 in General

Received this mail from Flowvps @Brendan (thanks for the heads up, and the message underscores once again the importance of backups, and backups of backups! )

Hello,

If you are receiving this email, VPS services you have with us located on NODE5 have had their data unrecoverably lost. Recovery attempts have been made however the data array has become unrecoverably corrupt.

It is unfortunate this email needs to be sent however it is necessary you know as soon as possible to implement your disaster recovery plan.

Further updates will be provided about redeployment of services and 3 months of service credits will be added to all accounts effected.

Sorry for any inconvenience this may cause.

Regards,
Brendan

edit: corrected typos

Tagged:

Comments

  • NeoonNeoon OG
    edited August 2020

    Well, I did not get that message but my monitoring says ded beef.

    Thanked by (1)bdl
  • my microlxc and my other Flowvps VPS both down. Thank fully I got Proxmox backup server beta working (at another site) last night :smiley:

  • NeoonNeoon OG
    edited August 2020

    @bdl said:
    my microlxc and my other Flowvps VPS both down. Thank fully I got Proxmox backup server beta working (at another site) last night :smiley:

    Yea I noticed, as soon I have confirmation, will see.

  • bdlbdl OG
    edited August 2020

    @Neoon said:

    @bdl said:
    my microlxc and my other Flowvps VPS both down. Thank fully I got Proxmox backup server beta working (at another site) last night :smiley:

    Yea I noticed, its DEAD, very bad.

    It's probably all @Brendan needs with the covid-related lockdown in Melbourne - hope he has luck getting it all back on track, was very impressed with the proactive notice

    I'm getting old, thinking I locked myself out of my microLXC due to borking a whitelist, but then realised it was hosted at FlowVPS :lol: need more beer

    Thanked by (3)vyas vimalware MaxKVM
  • vyasvyas OGRetired

    @SmallWeb Melbourne service seems to be okay- though I got multiple downtime alerts last night IST (GMT + 5.5)

    Thanked by (1)bdl
  • edited August 2020

    I'm glad I've got QuantumCore to fall back on, for my sole Oz client. My DR plan has a few wee niggles (email forwarders) but ain't too bad considering the e-commerce site is running under CWP. (The inbuilt CWP backups are crap!)

    Time on Error: 3 hours 2 minutes 31 seconds

    Used my tertiary backup (Online/Scaleway) to retrieve the files from other side of the globe, this time but I think I need to re-instate the Oz->Oz primary backup. ;)

    Haste ye back, @Brendan

    (A special shoutout to @QuantumCore for being there when needed the most - previous time being HostDoc demise. I'm sure the same won't happen here! ;) )

    It wisnae me! A big boy done it and ran away.
    NVMe2G for life! until death (the end is nigh)

  • MichaelCeeMichaelCee Hosting ProviderOGServices Provider

    @vyas said:
    @SmallWeb Melbourne service seems to be okay- though I got multiple downtime alerts last night IST (GMT + 5.5)

    Wouldn't know what node I am on, but no email received. Last remote backups Friday 21st.

    Would be interested to know what's setting your monitor off as mine is fine:
    https://status.smallweb.net/report/uptime/7ca1e97ad9b8383cb61fc2d43b60ecb5/

  • edited August 2020

    Time to start again..

    All VPS services are now ready to be reprovisioned.

    Thanked by (1)MichaelCee

    It wisnae me! A big boy done it and ran away.
    NVMe2G for life! until death (the end is nigh)

  • MichaelCeeMichaelCee Hosting ProviderOGServices Provider

    Very unfortunate. Best wishes to all.

  • YmpkerYmpker OGContent Writer

    That sucks :/ Best wishes to those affected! Hopefully things'll work out quickly :)

  • Hello,

    Thank you for your patience while this is resolved.

    All VPS services are now ready to be reprovisioned.

    Please use the service control area to reinstall your VPS to gain access, this will give you a clean install and you will be issued with a root password. If you'd prefer to do a custom install via ISO that option is also available.

    Please reach out via a support ticket if you need any help with the above task or if you have any other issues.

    Over the next week any services affected by this will have their next due date pushed forward 3 months, no action is required on your side.

    Regards,
    Brendan

    Thanked by (1)Ympker
  • My idling VPS is dead. Meh.

  • InceptionHostingInceptionHosting Hosting ProviderOG

    @MonstaAU said:
    My idling VPS is dead. Meh.

    Did you backup your idle?

    you can restore with touch /nothing

    https://inceptionhosting.com
    Please do not use the PM system here for Inception Hosting support issues.

  • vyasvyas OGRetired
    edited August 2020

    Anybody run YABS or any other BM on the new and revived FPS from this node?
    I got the following speeds (ticket of course being raised) but just checking reg disk speeds NVMe

    fio Disk Speed Tests (Mixed R/W 50/50):
    ---------------------------------
    Block Size | 4k            (IOPS) | 64k           (IOPS)
      ------   | ---            ----  | ----           ---- 
    Read       | 79.96 MB/s   (19.9k) | 341.99 MB/s   (5.3k)
    Write      | 80.17 MB/s   (20.0k) | 343.79 MB/s   (5.3k)
    Total      | 160.13 MB/s  (40.0k) | 685.79 MB/s  (10.7k)
               |                      |                     
    Block Size | 512k          (IOPS) | 1m            (IOPS)
      ------   | ---            ----  | ----           ---- 
    Read       | 295.00 MB/s    (576) | 334.80 MB/s    (326)
    Write      | 310.67 MB/s    (606) | 357.09 MB/s    (348)
    Total      | 605.67 MB/s   (1.1k) | 691.90 MB/s    (674)
    
  • bdlbdl OG
    edited August 2020

    @bdl said:

    @Neoon said:

    @bdl said:
    my microlxc and my other Flowvps VPS both down. Thank fully I got Proxmox backup server beta working (at another site) last night :smiley:

    Yea I noticed, its DEAD, very bad.

    It's probably all @Brendan needs with the covid-related lockdown in Melbourne - hope he has luck getting it all back on track, was very impressed with the proactive notice

    @vyas said:
    Anybody run YABS or any other BM on the new and revived FPS from this node?
    I got the following speeds (ticket of course being raised) but just checking reg disk speeds NVMe

    > fio Disk Speed Tests (Mixed R/W 50/50):
    > ---------------------------------
    > Block Size | 4k            (IOPS) | 64k           (IOPS)
    >   ------   | ---            ----  | ----           ---- 
    > Read       | 79.96 MB/s   (19.9k) | 341.99 MB/s   (5.3k)
    > Write      | 80.17 MB/s   (20.0k) | 343.79 MB/s   (5.3k)
    > Total      | 160.13 MB/s  (40.0k) | 685.79 MB/s  (10.7k)
    >            |                      |                     
    > Block Size | 512k          (IOPS) | 1m            (IOPS)
    >   ------   | ---            ----  | ----           ---- 
    > Read       | 295.00 MB/s    (576) | 334.80 MB/s    (326)
    > Write      | 310.67 MB/s    (606) | 357.09 MB/s    (348)
    > Total      | 605.67 MB/s   (1.1k) | 691.90 MB/s    (674)
    > 

    here you go, just ran now

    fio Disk Speed Tests (Mixed R/W 50/50):
    ---------------------------------
    Block Size | 4k            (IOPS) | 64k           (IOPS)
      ------   | ---            ----  | ----           ---- 
    Read       | 77.46 MB/s   (19.3k) | 406.85 MB/s   (6.3k)
    Write      | 77.67 MB/s   (19.4k) | 408.99 MB/s   (6.3k)
    Total      | 155.14 MB/s  (38.7k) | 815.85 MB/s  (12.7k)
               |                      |                     
    Block Size | 512k          (IOPS) | 1m            (IOPS)
      ------   | ---            ----  | ----           ---- 
    Read       | 337.10 MB/s    (658) | 412.93 MB/s    (403)
    Write      | 355.02 MB/s    (693) | 440.43 MB/s    (430)
    Total      | 692.12 MB/s   (1.3k) | 853.36 MB/s    (833)
    
  • vyasvyas OGRetired

    @bdl said:

    It's probably all @Brendan needs with the covid-related lockdown in Melbourne

    here you go, just ran now

    > fio Disk Speed Tests (Mixed R/W 50/50):
    > ---------------------------------
    > Block Size | 4k            (IOPS) | 64k           (IOPS)
    >   ------   | ---            ----  | ----           ---- 
    > Read       | 77.46 MB/s   (19.3k) | 406.85 MB/s   (6.3k)
    > Write      | 77.67 MB/s   (19.4k) | 408.99 MB/s   (6.3k)
    > Total      | 155.14 MB/s  (38.7k) | 815.85 MB/s  (12.7k)
    >            |                      |                     
    > Block Size | 512k          (IOPS) | 1m            (IOPS)
    >   ------   | ---            ----  | ----           ---- 
    > Read       | 337.10 MB/s    (658) | 412.93 MB/s    (403)
    > Write      | 355.02 MB/s    (693) | 440.43 MB/s    (430)
    > Total      | 692.12 MB/s   (1.3k) | 853.36 MB/s    (833)
    > 

    thanks, just learnt from colleagues at woork from AU that they are in Stage 4 lockdown. Maybe will hold off on the ticket for now and let it idle a bit more

    Thanked by (1)bdl
  • BrendanBrendan Hosting ProviderOG

    @vyas said:

    @bdl said:

    It's probably all @Brendan needs with the covid-related lockdown in Melbourne

    here you go, just ran now

    > > fio Disk Speed Tests (Mixed R/W 50/50):
    > > ---------------------------------
    > > Block Size | 4k            (IOPS) | 64k           (IOPS)
    > >   ------   | ---            ----  | ----           ---- 
    > > Read       | 77.46 MB/s   (19.3k) | 406.85 MB/s   (6.3k)
    > > Write      | 77.67 MB/s   (19.4k) | 408.99 MB/s   (6.3k)
    > > Total      | 155.14 MB/s  (38.7k) | 815.85 MB/s  (12.7k)
    > >            |                      |                     
    > > Block Size | 512k          (IOPS) | 1m            (IOPS)
    > >   ------   | ---            ----  | ----           ---- 
    > > Read       | 337.10 MB/s    (658) | 412.93 MB/s    (403)
    > > Write      | 355.02 MB/s    (693) | 440.43 MB/s    (430)
    > > Total      | 692.12 MB/s   (1.3k) | 853.36 MB/s    (833)
    > > 

    thanks, just learnt from colleagues at woork from AU that they are in Stage 4 lockdown. Maybe will hold off on the ticket for now and let it idle a bit more

    Please raise one, I'll take a look. But lower than it should be.

    Thanks everyone for being good about this, not an ideal situation for anyone.

    Thanked by (1)bdl
  • vyasvyas OGRetired

    @Brendan said:

    Please raise one, I'll take a look. But lower than it should be.

    Thanks everyone for being good about this, not an ideal situation for anyone.

    I am okay to wait @Brendan, so that you can fight the bigger fires first. Will raise the ticket in due time.
    Cheers

    Thanked by (1)bdl
Sign In or Register to comment.