Internet Worldview in Messaging World

Martin Sustrik, June 8th, 2010

When I write or speak about "messaging Internet-style" (see e.g. here), people tend to agree that Internet principles like no central control or routing around failure are desirable but often fail to understand how such a thing can be achieved in the world of messaging. I've thus decided to write a short story about programmer called Joe that would hopefully make the concept easier to grasp. Enjoy!

Joe has an idea. He believes people have a need for a service that sums two integers. It's just too much trouble to do it yourself! There's definitely a commercial value in providing such service! Therefore he decides to write a client/server application to do exactly that. He chooses to use 0MQ to do the networking stuff, because it allows him to implement it in 10 minutes. His server simply receives two integers from a client, sums them and sends the result back to the client. It's an "addition server":

#include <zmq.hpp>

int main ()
    //  Initialise the socket.
    zmq::context_t ctx (1);
    zmq::socket_t s (ctx, ZMQ_REP);
    s.bind ("tcp://*:5555");

    while (true) {

        //  Get a request from client.
        zmq::message_t request;
        s.recv (&request);
        int *request_data = (int*) ();

        //  Prepare the reply.
        zmq::message_t reply (sizeof (int));
        int *reply_data = (int*) ();

        //  Do the business logic.
        reply_data [0] = request_data [0] + request_data [1];

        //  Return the reply to the client.
        s.send (reply);

    return 0;

The client gets two integers from the command line, sends them to the server to perform addition, then it prints out the result:

#include <stdlib.h>
#include <stdio.h>
#include <zmq.hpp>

int main (int argc, char *argv [])
    //  Parse the command line.
    int number1 = atoi (argv [1]);
    int number2 = atoi (argv [2]);

    //  Initialise the socket.
    zmq::context_t ctx (1);
    zmq::socket_t s (ctx, ZMQ_REQ);
    s.connect ("tcp://");

    //  Create and send the request.
    zmq::message_t request (2 * sizeof (int));
    int *request_data = (int*) ();
    request_data [0] = number1;
    request_data [1] = number2;
    s.send (request);

    //  Receive the reply.
    zmq::message_t reply;
    s.recv (&reply);
    int *reply_data = (int*) ();

    //  Print out the result.
    printf ("%d+%d=%d\n", number1, number2, reply_data [0]);

    return 0;

When Joe tests his application on his box he runs just one server and one client instance. It looks something like this:


However, his idea is to deploy the client applications to the users and run the server on his mainframe. The desired deployment looks like this:


Normally, the story would end here. Users are happy with Joe's services. Joe is making a lot of money on "addition services" and have bought a yacht.

However, this article intends to explain what "messaging Internet-style" means. So we'll have a look what happens as the market for "addition services" grows.

As the number of customers grows, Joe's mainframe is no longer capable of handling the load. What he needs now is to spread the load among multiple boxes. To solve the problem he installs a "queue", a component that clients can connect to and that load balances the requests among several computing nodes:


The nice thing about the design is that Joe can now add and remove services on the fly, depending on the actual demand for the computing power. This becomes really handy when he moves his business to a cloud provider and suddenly gets an option to add and remove virtual boxes as needed.

Once again, the story could have ended here if not for an evil corporation called AdditionWare Inc. AdditionWare Inc. realises the business potential in the market for addition services and moves in competing with Joe's service.

Meet user Jim. So far he have been using Joe's addition service. At this point he has to choose. He can either continue to use Joe's service which is cheap but somewhat unreliable or switch to AdditionWare Inc. which is way more expensive but has 99.99999% uptime.

Jim is a small user with no money to waste, so he prefers to stay with However, when Joe's service is temporarily unreachable or unresponsive he doesn't mind spending few bucks to fail over to AdditionWare's service. What he does is that he configures his client application to use with priority 1, and with priority 2. Jim's topology now looks like this:


An yet once again this would be the end of the story if the passion of adding two integers wouldn't become a favourite pastime for millions, skyrocketing the usage (and revenues) to ludicrous levels. More and more addition service providers pop up and users have to maintain still longer lists of available service providers, check for discounts available at the moment, update their priorities accordingly etc.

At this point AddCheapli! plc comes with a brand new business model: For a fraction of cost of a single addition request they'll forward users' requests to the service that is cheapest at the particular moment. Their ambition is to become an exchange in addition services. They even provide an ability to place "limit orders" when the request is processed only if the price drops below user-specified limit:


Now the users connect to AddCheapli!'s service and don't have to care about finding out which addition service provider offers the best price at the moment.

However, user Jim is still concerned about reliability. AddCheapli! have been unavailable on couple of occasions and it certainly doesn't achieve AdditionWare Inc's 99.99999% uptime. Thus Jim decides to use AddCheapli! as his primary addition service provider, but to fail over to AdditionWare Inc. when it becomes unavailable:


The story doesn't end here. The infrastructure will evolve further, but we'll leave our heroes at this point and put down some observations:

  1. When Joe had written his client/server application he haven't done so with the idea of creating an ecosystem for addition services. He've used request/reply pattern (as implemented in 0MQ) just because it made implementing client/server networking stack easy. However, the ecosystem — and whole market for addition services — have emerged from the fact that suddenly there was an easy way to integrate third parties into the infrastructure with no central authority to keep users locked-in or decide who's allowed to join the ecosystem and who's not.
  2. The infrastructure grew from edges to the center. At the beginning the Internet was used only as a dumb pipe to connect users to Joe's addition service. As the number of users and service providers grew, new intermediary nodes were added doing all kinds of smart routing and turning Internet from a dumb pipe into a smart network, so to say to a "global cloud" for addition services.
  3. This story focused solely on "request/reply" messaging pattern. However, similar stories can be we written about other messaging patterns, such as "publish/subscribe" or "pipeline".

The lessons learned are:

  1. Solve the simplest possible problems. Allow the solution to scale. For example, if 0MQ would require a broker in the middle, Joe wouldn't used it at all. Having to configure and run a broker means just too much trouble for a person writing a client/server application. On the other hand, if the solution doesn't scale, Joe's addition service would remain what it is — a small service in a big Internet world — and would never turn into a global ecosystem.
  2. Grow from edges to the center. If AddCheapli! addition service exchange would be established first with the idea of creating a global market for addition services, it would never succeed. There would be no commercial incentive for people to write addition services and join the AddCheapli's network. Everyone would be better off letting the users connect directly to their service. The reason why AddCheapli! succeeded was that there was already a large number of edge nodes (service providers and users) in the network.
  3. Use well-designed messaging patterns. While the overall architecture described in the story seems obvious, almost trivial, it's not. The versatility it provides is grounded in the fact that "request/reply" pattern was deliberately designed to be versatile and scalable. There are other widely-used messaging patterns that don't have these properties. One example of such anti-pattern is called "multicast bus". It is often used in stock-trading industry. For those not familiar with it, it simply means that each message is sent to everyone else within the network. Multicast bus pattern works only with exactly one intermediary node. There's no way to scale it any further.
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License