NAT + Anycast == Love ?

mikho

After the offer for Anycast was posted here on LES, it got me thinking if this would be an interesting idea to implement into the NAT world?

I can see a positive thing about having numerous (cheap) locations around the globe, all accessible via anycast.

As NAT, a single setup adding the shared IPv4 would make it easy to maintain in that aspect.
I would like the community input on pros and cons, would you be interested in using Anycast?

Please, sit down in groups of three and discuss ....

tetech

Pro: for a low-end project, anycast can cost several times more than the VPS's themselves, which makes it pretty hard to justify. If you can share an anycast IP among a lot of people and have a reasonable traffic allowance due to bulk purchase, it could push the price to a viable point... depends what the price actually is.

Con: a big use for anycast is own DNS, and since that's UDP it's not playing nicely with NAT. A second issue is redundancy. Since people will not have control of the IP, you'd need to handle any failover for them. Potentially something annoying is if a single container fails but the host is still up, the anycast IP is pointing into a black hole for that container owner - perhaps some HAProxy magic needed too?

Might it also bring some restrictions on location, i.e. for a single anycast IP, I guess the people on that IP would need the same pool of VPS locations? So the "pick any X locations" might not work so good.

But, if I could get a bundle of anycast-enabled VMs at a reasonable price increment, I would be very interested.

foxone

YES PLEASE

seanho

This would be awesome! I never considered that it might actually work.

jureve

Very interesting!

willie

It's a cute idea but tetech's reservations are well taken, and also NAT services are frequent DDOS targets and generally low end enough (many many containers per hardware node) that trying to build high availability services from them seems a bit misguided.

mikho

@tetech said:
Pro: for a low-end project, anycast can cost several times more than the VPS's themselves, which makes it pretty hard to justify. If you can share an anycast IP among a lot of people and have a reasonable traffic allowance due to bulk purchase, it could push the price to a viable point... depends what the price actually is.

As you say, the traffic is what will be the large portion of the cost for the service.
Really not sure on how to calculate this per customer. Yet....

@tetech said:
Con: a big use for anycast is own DNS, and since that's UDP it's not playing nicely with NAT. A second issue is redundancy. Since people will not have control of the IP, you'd need to handle any failover for them. Potentially something annoying is if a single container fails but the host is still up, the anycast IP is pointing into a black hole for that container owner - perhaps some HAProxy magic needed too?

One idea would be to only open up a couple of ports for the Anycast IP.
Example: only port 80&443 is open (using haproxy to forward to the vps itself).

@tetech said:
Might it also bring some restrictions on location, i.e. for a single anycast IP, I guess the people on that IP would need the same pool of VPS locations? So the "pick any X locations" might not work so good.

This would require a container on each location added to the anycast service.
This would require that the user really knows how to setup their service to work on that many locations.

@tetech said:

But, if I could get a bundle of anycast-enabled VMs at a reasonable price increment, I would be very interested.

Thats part of the reason I opened this thread, to get use cases where it would be useful.

@willie said:
It's a cute idea but tetech's reservations are well taken, and also NAT services are frequent DDOS targets and generally low end enough (many many containers per hardware node) that trying to build high availability services from them seems a bit misguided.

Do you speak from experience?
I can’t say I that I’ve had that many occurances of DDoS during my years as a NAT provider.
I would say that the low resources makes them less of a target because there are no single large community running on a 128/256mb container.

I try to build all of my nodes with raid10 if hdd or at least mirrored ssd drives.
It has never been marketed as being top of the line, latest cpu type of service but it should be stable enough to run a small wiki/wordpress site.
And the cheap price makes it possible to have a failover in case of node issues.

tetech

mikho said: Thats part of the reason I opened this thread, to get use cases where it would be useful.

For me the interesting anycast use cases are (a) custom/own DNS, (b) low-latency and/or geo-targeted HTTP(s), and (c) faster failover than DNS in the event a server goes down, and it also reduces the number of "premium DNS lookups" on some managed DNS plans.

mikho said: One idea would be to only open up a couple of ports for the Anycast IP.
Example: only port 80&443 is open (using haproxy to forward to the vps itself).

This would be fine for the above-mentioned use cases, since DNS servers are not considered on a NAT service anyway. For host-level failures, I would assume you'd take care of removing the host from the list of anycast nodes so traffic goes to the next-closest.

From my (customer) perspective the main thing I'd like to know is how to handle container outages. Example scenario: I have some planned downtime on my US-West VPS due to re-imaging or upgrading the OS, so I don't want anycast traffic coming to that VPS because there is no HTTP server running. So there needs to be some way either for me to manually configure that, or your HAProxy is configured with multiple (lower-priority) backends pointing to other locations, US-East etc.

mikho said: This would require a container on each location added to the anycast service.

Yes, so the customer does lose some flexibility in choosing any X locations from your large list (to suit their target audience), and potentially has to buy/maintain more than they need. Not a major problem, but a consideration. Depending on cost (IP vs bandwidth) you could potentially offer a few IPs for "2 continent package", "3 continent package", "4 continent package".

ETA: Thank you for raising the idea, I'm pleased to have the discussion! I don't want to seem negative, just analytical.

Daniel

I'd be interested in anycast IPv6, if you ever offer that! For all my services with you I only ever use the IPv6 address for inbound connections, I don't use the shared IPv4.