When I was an adolescent, maybe 18 years old, I was hit over the head with the realization that we have no say in our survival as species. Evolution is going to progress in a way that maximizes multiplication and our beliefs, our cleverness, our science, our will to sacrifice personal wellbeing to the benefit of all is not going to change that a bit.
I was never able to communicate the horror of being at the mercy of cold and uncaring, non-sentient in fact, forces of nature to anyone else. At the time I was living in Bratislava and hanging out with the local bohème. Getting terrified by theoretical speculations was not exactly their cup of tea. And surely, given that the evolution progresses at generational pace, the problem, however troubling in theory, is not terribly urgent, right?
What I failed to communicate was the underlying structural pattern. The fact that the idea generalizes. If all self-optimizing feedback loops suffer from the problem, it's not just the evolution that doesn't care about our will and progresses in an unpredictable direction. It's also the economy. It's all the diverse feedback loops that the fabric of society is made of.
I had this terrifying insight, at a quite an early age, and I was left alone with it. It was like being the only passenger on Titanic who have noticed the iceberg. The one who you see running around and screaming and getting nobody's attention.
One would think I have got some relief from reading scientific literature. Scientists are clever. They have all these deep insights. They do see patterns in things. And they are not afraid of theorizing! Well, I have read quite a lot of evolutionary biology. And evolutionary biologists haven't convinced me they see the danger. Presumably, they were too sober to write about such philosophical and hand-wavy topics. What about the economists then? They wouldn't mind a bit of hand-waving, would they? I found nothing. I guess they were too busy inventing new financial instruments. Or, more probably, both evolutionary biologists and economists have indeed seen the issue but haven't considered it to be a problem. It's just the natural order of things, isn't it? Then philosophers maybe? This seems to be exactly the kind of topic a philosopher would be interested in. I have to admit I haven't read much philosophy. I've got disheartened by too much inconsequential mumbling in the books. As a friend of mine, a professional philosopher, desperately cried after a long discussion we've had: "Sústrik, you don't care about philosophy! You care about truth!" Anyways, even today I am not aware of a philosopher seriously tackling this problem. Daniel Dennett seems to have all the necessary mental machinery but yeah, he's working on his congnitive science stuff.
Then, after being in this solitary confinement for more than twenty years, I've read Scott Alexander's "Meditations on Moloch". The essay dealt exactly with the problem I was dealing with. It was very explicit about it and illustrated it with a bunch of nice and easy-to-understand examples. You can't imagine how relieved I was! There was someone out there who actually did care!
I am writing all the above to make it clear that whatever objections I am going to express they are of little importance. The major feat of acknowledging the topic was already accomplished. We are already speaking! Oh my boy! That's like gaining 100 points in a game. If my objections are like losing 1 point, well, all right, but guess how much I care.
Back to the Moloch essay though. At some point, where one would expect him to be finished with examples and to move to solutions, Alexander gets all metaphorical, dark and lovecraftian. He invokes eldritch gods and unspeakable horrors from beyond space and time. The second part is basically a piece of prose poetry dealing with this disturbing topic. It may nicely express how I feel about it, but it does very little to advance the solution to the problem.
Yes, I do understand. The problem is hard. It's not as if I didn't know that. And where there is no solution in sight and all you are trying to do is to communicate the gravity of the issue then maybe poetry is the right tool for the job.
Still, as a practical-minded person, I crave solutions.
Enter Eliezer Yudkowsky's new book, "Inadequate Equilibria".
Reading the book was really a thrill. It covers the the same ground as Alexander's essay but it goes into much more depth. The examples, especially the one about treatment of the short bowel syndrome, are much longer and discuss the phenomena in question from different points of view. The book even introduces new tools to tackle the problem. The concept of "inexploitability" is something that both evolutionary biologists and economists have understood for a long time but nobody have devised such a sweet and succinct way to communicate it to the laymen. And the inexploitability concept, I think, is up to the task. I, for one, am definitely going to add it to my mental toolbox.
I am not that sure about the other new concept, "inadequacy", mostly on the ground that it's far less intuitive than the former one. However, I haven't digested it fully yet, so I may be wrong. Give me some time.
I any case, the very fact that someone started to work on the problem and devised some tools was amazing. I was thrilled. What was coming up next? I was turning the pages quickly. And then, at some point, Yudkowsky veers off to epistemology. I was like: "What? Where are my inadequate equilibria?" I've turned pages even faster only to discover that inadequate equilibria are not going to come back.
After finishing the book I was confused. I felt like if a girl was flirting with me entire evening and then, suddenly, left with a different guy. I felt cheated.
Then I re-read the first chapter and realized that everything was the way it was on purpose. It's not like Yudkowsky is an incoherent idiot who cannot stick to his topic after all. He was aiming at epistemology from the very beginning. The first sentence of the book makes it clear:
This is a book about two incompatible views on the age-old question: “When should I think that I may be able to do something unusually well?”
He wants to identify the cases where he can do better than established institutions and the whole discussion of inexploitability/inadequacy is just the tooling used to answer that question. It may not be what I would prefer to read, but it's intellectually consistent.
And that brings me to a more general observation. It's something that always made me feel slightly uncomfortable about the entire Lesswrong crowd and the related rationalist community. However, I never followed it too closely, so take what I'll say with a grain of salt. It's very much possible that the flaw I see is limited to just few individuals.
The explicit goal of the rationalist community is to improve participants' reasoning skills, to make them aware of their biases and so on. Great. Nobody can argue with that. But if you take a step back the perspective changes: What's the point of ejecting a tiny sliver of population into the intellectual stratosphere? And if there's any grand cause behind it, other then having fun, what exactly is it? Why not rather focus on improving everyone's cognitive skills, even the skills of those that are less gifted, less interested, those lazier and those that have too much other work to do? Why not do what Amazing Randi does? Why not do what the new atheists do? Why not debunk some charlatans and faith-healers? It may not be very sexy but maybe it would be more effective in the end than the elitist project that almost by definition focuses on the top 0.1% of the population.
Let me be super blunt here: Jews may have had the largest intellectual firepower of all the nationalities of Central Europe in 1925. By 1945 there were almost no Jews left in Central Europe. In short: A handful of intellectual supermen isn't going to save us.
Now, don't get me wrong. I said "slight discomfort" for a reason. A lot of folks in rationalist community seem to be interested in effective altruism, which definitely counts as doing something for everyone. There may also be people who are also participating in the skeptic and the atheism-promoting communities. Doing the rationalist thing isn't harmful by itself. Just don't forget, while taking another dose of nootropics, about all the people you are leaving behind. Have an explicit plan of how to make them follow. And no, trickle-down rationalism is not going to work. If there's a bright guy in West Virginia who gets enlightened by the Sequences he's not going to stay at home and share the light with his friends and neighbours. He's going to pack his luggage and move to Silicon Valley.
Enough of this moralistic rattle though. Let's get us back to the inadequate equilibria problem.
Multiple people have reviewed the book. Here's Scott Aaronson. He's a very clever person but, sadly, he doesn't offer any more insight on the problem of suboptimal equilibria. Here's, again, Scott Alexander with his review that made me laugh at several places. It makes some good points about the epistemological part of the argument but, unfortunately, there's nothing new on how to deal with inadequate eqilibria. Finally, Robin Hanson reviews the book and dedicates two sentences to a possible solution, namely, prediction markets. I am not an expert on prediction markets and two sentences aren't enough to explain his position in full. The first things to discuss though would be: Why do you believe that prediction markets would dissolve the old suboptimal equilibria and won't, at the same time, create new, maybe even worse ones? And how do you know that? Is there any meta-theory that would allow us to evaluate such claims? But all that being said, Hanson is a single person I am aware of who actually sticks his neck out and dares to make a proposal. He wins 100 points.
So, to finish this review let me make some proposals of my own.
First, let's first look at some real world examples and try to find out how suboptimal equilibria tend to play out in practice. I mean, you really don't have to sweat blood to come up with that approach.
I am not a historian or a sociologist, but my guess is that suboptimal equilibria tend to accumulate over time. As time progresses there are more and more non-functional institutions, misaligned incentives etc. Then, at some point, the malfunction is so grave that it leads to major upheaval which resolves the problems by brute force, simply by smashing the entire institution apart.
As a toy example, let's look at one scenario introduced in Yudkowsky's book. Craigslist is a website to connect sellers and buyers in San Francisco Bay Area. It dates back to 1995 and thus it has very basic functionality. Why doesn't (hypothetical) Danslist with better and more convenient features replace it? asks Yudkowsky. Because the buyers are on Craigslist, he answers, and the sellers want to be where the buyers are. And the other way round. The sellers are on Craigslist and the buyers want to be where the sellers are. This way, the inadequate equilibrium is maintained.
Now imagine for a second that Red Army invades San Francisco. They bomb the hell out of it. They send the enemies of the people (venture capitalists and programmers) to Gulags and everyone else has to work in fish-canning factories. Raw oil is being pumped into the Bay. Internet is dismantled and replaced by officially sanctioned Pravdanet. You get the idea.
Now think about what happens after twenty years, when the Bay Area is finally liberated. Some people will still have memories of Craigslist, but the conviction that "everybody is on Craigslist" has evaporated during those dark communist years. When Craigslist it relaunched, in parallel with Danslist, there's no big reason to prefer one to another. However, Danslist has better features. So people crowd to Danslist. Inadequate equilibrium is solved.
Does the same thing happen in the real world though?
Well, one can claim that much of the inefficiencies of absolute monarchy were solved by French revolution and the following sequence of republics, empires, revolutions and wars. By the year 1900 it's hard to find something that screams "I am a thing of the past!" louder than the salon of duchesse de Guermantes.
The old suboptimal equilibria were dismantled and new ones were created. We are in the world of Verdurins now. In the age of unhindered capitalist expansion. Runaway expansion (And who says we have to limit ourselves to equilibria? Runaway processes are just as interesting!) have crossed the national boundaries and resulted in the enormous, unsustainable colonial empires.
And once again, one can argue that WWI, with WWII as its follow-up, have ended that epoch. In 1945, the colonial empires were, for all practical purposes, dead. It may have taken a decade or two for them to fully decompose but the days of Leopold II-style colonialsm were over.
Am I suggesting that the above is actually true? Not at all. However, it's a testable hypothesis. Coming up with metrics of societal dysfunction may not be trivial but it is, presumably, a doable task. Once we have the metrics we can measure how the dysfunction varies with time. And we can ask questions. Does the dysfunction increase during the periods of social rest? Does it decrease in the periods of social chaos?
Or we can try to be even more humble. We can start by making a catalog of known real-world cases of inadequate equilibria. You identify one, give it a number, then describe it from historical perspective: How did it come to be? How did it survive? What challenges have it faced? How was it finally dismantled?
With such a catalog we can start asking quantitative questions: How does the distribution of inadequate equilibria lifetimes look like? Is there anything special about the long-lived ones? Is there a prevalent cause for their dissolution? Are there any measures meant to end them that are never successful? And so on.
OK, you get it. Now let's move to a different idea. This one is much less down-to-earth, much more meta and it requires some background knowledge. But it the end, I think, it's not completely incomprehensible.
Let's start with the case of the rats from Scott Alexander's essay:
Suppose you are one of the first rats introduced onto a pristine island. It is full of yummy plants and you live an idyllic life lounging about, eating, and composing great works of art. You live a long life, mate, and have a dozen children. All of them have a dozen children, and so on. In a couple generations, the island has ten thousand rats and has reached its carrying capacity. Now there’s not enough food and space to go around, and a certain percent of each new generation dies in order to keep the population steady at ten thousand. A certain sect of rats abandons art in order to devote more of their time to scrounging for survival. Each generation, a bit less of this sect dies than members of the mainstream, until after a while, no rat composes any art at all, and any sect of rats who try to bring it back will go extinct within a few generations. In fact, it’s not just art. […] If one sect of rats altruistically decides to limit its offspring to two per couple in order to decrease overpopulation, that sect will die out, swarmed out of existence by its more numerous enemies.
The rats are doomed to live in eternal darkness, with no art or music to brighten their days, right?
Well, that's what I though once. Then, few years after I've got that terrifying insight on the nature of our universe, someone asked me which recent scientific discovery I consider to be the most impactful. Very much to my own surprise, I replied: "Genetic engineering. We can finally give evolution a black eye."
The idea was extremely raw, it was hardly fit to be made into anything practical, but there was already a seed of something sensible there. The fight was maybe not as hopeless as it seemed!
It was only later that I realized that we don't need genetic engineering at all. To make a knot on evolution's respiratory pipe the only thing we needed was to steer it at the meta-level. And — what a happy coincidence! — we already had the tools for that for millennia!
Consider Alexander's rats. They have already evolved the ability to mimic each other. If they did not they would not be able to absorb artistic styles and create new art.
With the ability to mimic, memetic evolution enters the picture. Same way the genes compete for the space in bodies, the memes compete for the space in brains.
Reasoning about evolution is hard (see Orgel's second rule) but reasoning about concurrent evolution at two different levels (genetic and memetic) is just insane. However, in this particular case, we can make our lives easier by considering the memes to be be symbionts, whether parasitic or mutualistic, very much like bacteria, except that they lack a physical body.
What would we except to happen? Would the memes evolve to be beneficial to the rats? Would they hurt them? Well, again, reasoning about anything that has to do with evolution is hard, but the general pattern, at least for bacteria, seems to be that those that propagate with the host (from parent to child) tend to be mutualistic, i.e. beneficial to the host, while those that propagate horizontally, between genetically unrelated individuals, tend to be parasitic, i.e. harmful to the host.
It's not hard to imagine why. If the only place for the bacterium to propagate to are the host's children then it is positively interested in the host surviving and propagating. Hurting the host makes no sense because it hurts symbiont's own progeny. This seems to be the case of mitochondria, erstwhile bacteria who entered the eukaryotic cell some two billion years ago and have in the meantime lost the ability to survive on their own. In fact, a lot of their DNA is now stored in the cell's nucleus. The extreme case may be peroxisomes, different organelles with likely symbiotic origin that no longer have any DNA of their own.
If, on the other hand, bacteria propagate freely among the hosts it is perfectly all right for them to kill one host as long as they manage to propagate to other hosts before the original host dies. Horizontally propagating bacteria are, in most cases, much more virulent.
What does that mean for the memes? The memes that propagate vertically, from parents to children, are likely to be beneficial to the organism. So, for example, one would expect the orthodox Judaistic memeplex to contain a meme saying "Multiply and fill the Earth!" The memes which propagate vertically among the peers, say the pop culture, are much less likely to pay heed to the host's wellbeing. One interesting example can be found here, another one here.
So, in the end, meme saying "have just two children" can propagate among the rats even if it hurts their reproductive fitness. But, you say, isn't it in the meme's interest to make the host multiply so that it has more brains to propagate to? Well, no, because there are other memes it is competing with. If it allocates effort to promote multiplication of the host species the benefit would be reaped by all the memes, even those that haven't invested anything in the effort. Thus, the original meme would be less fit that other memes and would gradually die out. Tragedy of the commons can be sometimes exploited to our own advantage.
Still not getting it? No worries, evolutionary reasoning is a conter-intuitive mess. So once again: If there was a tribe of rats with ten children each it would easily outcompete those naive rats with just two children per family, correct?
No. Recall that we postulated that the two-kid meme spreads horizontally. The children of the fertile tribe would be eventually infected by the meme and will end up having two children. And given that the memetic transfer is much faster than genetic transfer it would happen long before the competing tribe would grow to any significant size.
I am not claiming that any of the proposals above is going to directly help to solve the problem of inadequate equilibria. My only intention was to show that the situation is not as hopless as it my seem at the first sight. Maybe, this way, I can gain next 100 points in the game for the survival of the human race.
December 4nd, 2017