HAProxy - forward client IP
After @Neoon kindly pointed me in the direction of haproxy, for reverse proxy use, I've nearly got a full setup. I'm struggling with the last piece of the puzzle, even after trying to comprehend the vast options in the documentation.
I'm trying to determine the original client IP. Straight http requests can be interrogated the through x-forward but this isn't much good for system monitoring, for example. Also, a control panel in a VM only sees access coming from the host IP.
Here's a snippet of what I've got, sorry for the formatting (need code tagging):
frontend webmin
bind *:10000
option tcplog
mode tcp
option http-server-close
option forwardfor header X-Client
acl tls req.ssl_hello_type 1
tcp-request inspect-delay 5s
tcp-request content accept if tlsacl host_ks.domain1.com_webmin req.ssl_sni -i ks.domain1.com
use_backend ks.domain1.com_webmin if host_ks.domain1.com_webminbackend ukc.domain1.com_webmin
server ukc.domain1.com 10.0.0.100:10000
mode tcp
Note: I had to change the host webmin port (not such a bad idea), as I couldn't get haproxy to ignore/pass it though. Below is currently commented out..
( in frontend webmin )
acl host_d3.domain1.com_webmin req.ssl_sni -i d3.domain1.com
use_backend d3.domain1.com_webmin if host_d3.domain1.com_webminbackend d3.domain1.com_webmin
server d3.domain1.com 178.123.123.90:10000
mode tcp
I have a further VM but left that out, for clarity.
All other aspects appear to be fine, with CSF redirecting various ports and VMs able to access the 'net.
Oh, and thanks again @Neoon
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
Comments
I'm not an HAProxy expert but I don't think there's an easy way to do that with straight TCP requests. There's just no provision for it in TCP. There is the PROXY protocol if your application supports it but I'm not sure what services do to be honest. I think I remember reading something about how you can put the backends on NAT'ed addresses and convince HAProxy to effectively spoof the source address. This completely isolates the backends though as HAProxy has to be the gateway otherwise the clients simply reply directly to the original IP. It's equally possible I'm delusional. I use HAProxy for some simple HTTP proxying and that's it.
If I understand your setup correctly, haproxy has nothing to do with this.
Haproxy just forwards tcp packet and everything within. If http inside tcp contains x-forward family header it will be there after haproxy forwards it. It means it is webmin that needs to pay attention to it.
Both his frontend and backend are running in tcp mode so ... ( confused) there's ... nothing injecting an x-forward-for header. Umm, is option forwardfor even valid in tcp mode? I just accepted you needed tcp mode and didn't question it.
You can add headers in the HTTP mode, but not in TCP.
Because the package contents cannot be manipulated since they are encrypted and the HAProxy has not the keys for it, it just checks if its valid TLS traffic, reads the SNI header and forwards it.
But it can manipulate the headers of the packages such as the IP address, which it does by default.
Either you terminate TLS on the HAProxy or you enable transparent mode, which is more complicated.
Free NAT KVM | Free NAT LXC
Thanks guys. I added the "http-server--close" and "forward for" later, to see if it made any difference, which it didn't for tcp mode. For completeness, a CWP installation (the other VM) on https port 2031 also displays the host IP as "your IP".
Is it perhaps the redirect used in interface startup and/or the redirects in CSF that aren't passing through the source IP?
Interfaces snippet:
plus this:
csf.redirect snippet:
If I run a phpinfo on the above VM (ports 443,80), then:
though X-CLIENT reports correctly for port 80
Haproxy was added after the interfaces/CSF setup, to (attempt to) allow accessing services using default ports. I do note that accessing CWP via the NAT port 43031 also displays the host IP, instead of the client one.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
Okay, so I misunderstood, I assumed the header must come from somewhere before it hits haproxy.
Why not use destination nat with default ports? Are there multiple instances of same service?
Can you use haproxy in http mode?
This appears to be the way forward. Goes off to Yahoo! (Google), after food.
if I understand correctly http mode only works for non-SSL.
I wish to be able to access the various VM services, without specifying obscure ports. For example: https://ks.domain1.com/nextcloud rather than the previously used https://ks.domain1.com:40443/nextcloud (which is still possible, of course).
Extending this further, say three VMs serving standard port 443 https, would not be possible without some form of redirection per VM.
Wholeheartedly admit my total lack of knowledge in this area.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
What do you mean only works for non-ssl?
You use SNI and set up ACLs to route traffic via the http header.
http should work if you terminate the TLS connections at HAProxy but that means HAProxy would need to have the keys.
Free NAT KVM | Free NAT LXC
Something like this:
frontend https
bind *:443 ssl crt /etc/haproxy/certs/haproxy.pem
( Can't get the formatting right but you get the point )
Sounds like another can of worms, getting LE certs in the right place, renewed etc. :-|
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
I'm used to doing it with dedicated certs but there are documents on how to set it up with HAProxy. Didn't look too bad to do but I, uh, might've cheated and used my webhost to create a wildcard. I'm waiting to see if when they renew it I have to refresh it my haproxy.pem. I'll know in June.
@AlwaysSkint okay, got you.
I believe you need nginx here - much more natural to terminate all https on nginx and forward to whatevs.
Things that are not http/s can be just dnatted since you don't need load balancing or smth I suppose.
@comi I started with nginx reverse proxy (not without some issues, in another thread *), then we can blame @Neoon for pointing me in the haproxy direction.
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
If you're not using the caching pieces of nginx I they're about the same and personally I find HAProxy config files easier to read.
Sorry, my bad, you can just as well terminate everything at haproxy, and setup the forward header. But everything behind haproxy need to read the header. If it can't you need to apply some jutsu for transparency.
So yeah, basically what @Neoon said.