Description of the problem:

There is a website which we want it to be Available even in the following situations:

  • The servers are not available (i.e. DDoS attack)

  • The cluster is not available

  • The domain is not available (i.e. DNS is down, the domain is in a blacklist, etc.)

  • The website is broken. It is available but is returning the wrong content (i.e. because of some bug or issue with the database)

What we need is kind of an external load balancing even between alternative domains.

However, there is something to consider. Having an external balancer is also a point of failure. If that is taken down ... What could be done in this situation? An idea would be that this balancer is "private", so each user or community has his own, installed on their private cluster, network, or even installed on their laptops?, ... but then how is the list of servers updated? And how do we stablish the trust on the list of servers?

I don't have much experience on HA nor proxies, so this is more of a research project.

The tasks will be:

  • Try to configure HAProxy as an external load/mirror balancing (is this doable?) and configurable as a proxy

  • "think" of a way to update the configuration on a secure manner

  • Implement a Proof Of Concept of 2

Looking for mad skills in:

haproxy blockchain dns

This project is part of:

Hack Week 16

Activity

  • about 2 years ago: jordimassaguerpla added keyword "dns" to Some research on HA and proxies and mirrors
  • about 2 years ago: jordimassaguerpla added keyword "blockchain" to Some research on HA and proxies and mirrors
  • about 2 years ago: jordimassaguerpla added keyword "haproxy" to Some research on HA and proxies and mirrors
  • about 2 years ago: jordimassaguerpla originated Some research on HA and proxies and mirrors
  • Show History

    Comments

    • jordimassaguerpla
      about 2 years ago by jordimassaguerpla | Reply

      While looking into the problem of how to "distribute" a list of server names in a trusted way, I came across blockstack and namecoin, which, simplifying a lot, are decentralized DNS based on Blockchain technology (like bitcoin).

      At the same time, looking on load balancing solutions I came across DNS round-robin, which in short is the DNS returning multiple A records for a domain name, that is, multiple IPs for one single name.

      Searching a bit more, I found out that modern browsers, when they look for a name, that is, they try to resolve it, they accept a list of IP addresses. Then, they will choose one and, if the connection fails, they will use another one from the list. In practice, this is DNS round robin HA. It does not balance the load but it works for High Availability.

      Thus, if we connect those ideas, my proposal to solve that problem would be to use blockchain technology, either namecoin or blockstack, as decentralized DNS, which will store multiple A records for one name, and configure that DNS on our system, so our browsers will do the round robin HA.

      This will fix most of the problems, except for the one that the website is broken (it returns something but it is not what you would expect).

    • jordimassaguerpla
      almost 2 years ago by jordimassaguerpla | Reply

      PR that adds multiple A records to the namecoin DNS.

      https://github.com/namecoin/nmcontrol/pull/121

    • jordimassaguerpla
      almost 2 years ago by jordimassaguerpla | Reply

      By using the previous PR, I was able to setup a "Decentralized High Availability" by:

      • Using namecoins to have a decentralized domain name, in this example jordia65.bit, which had multiple A records (2 ip addresses). I bought this bit domain in peername.com. See: https://namecha.in/name/d/jordia65

      • Starting 2 webservers in digital ocean with the IP addresses that jordia65.bit domain "links to"

      • Installing namecoind and namecoin-qt in order to have a local copy of the namecoin blockchain history

      • Setting up nmcontrol to query the namecoind daemon

      Then, the test was to:

      1- Use nslookup to look for jordia65.bit, and effectively, it returned 2 ip addresses.

      2- Use wget to get http://jordia65.bit. It downloaded the index.html from the first server in digital ocean.

      3- Shutdown the server.

      4- Use wget again and see how wget tells you that the first server is unavailable and tries up with the second one.

      5- Do the same test with firefox.

      So, it is possible to setup a Decentralized HA by using namecoins.

    • jordimassaguerpla
      almost 2 years ago by jordimassaguerpla | Reply

      Before doing all that, I tried setting up a proxy on a vm with apache2+mod_proxy. Then, in that same vm, I set up a DNS server with the jordia65.bit "zone" so that it returned 2 ip addresses.

      Then, I setup firefox to use that proxy and did the same tests as before, and I realized that this worked, meaning firefox was delegating the dns resolution to the proxy but if the first server was down, firefox was connecting to the second one.

    • jordimassaguerpla
      almost 2 years ago by jordimassaguerpla | Reply

      My conclusion is that we should be able to setup a decentralized HA by setting up namecoin+nmcontrol in a proxy, and then configuring our browser to use that proxy.

    Similar Projects

    This project is one of its kind!