The Internet Impossibility Theorem, 4/27

Why Decentralization Fails and Choke Points Emerge

Late in 1995, Robert Metcalfe predicted that under the pressure of increased traffic the Internet would soon collapse spectacularly. This did not happen, and Metcalfe famously ate his words.

But he was right. And this has dire implications for the libertopian dreams of decentralized finance powered by cryptography.

Metcalfe, who had invented Ethernet, understood the technical vulnerability of the decentralized architecture of the Internet. What saved the Internet from collapse was re-architecting it. What survived was no longer the Internet, in that it no longer was decentralized.

Keeping that in mind, my intuition is that centralized architecture will always emerge. Centralized architecture in turn creates choke points that permit control. I call this the Internet Impossibility Theorem: achieving a decentralized society is a task for culture, not for engineering.

In the original vision of a decentralized Internet, every request for information would travel through the network from one end to the other. Your computer asks for the information packets that make up this web page, and that request finds its way to a computer at Substack and back again. As described by Ed Krol, the Internet’s communication links and router-computers acted like a Pony Express, relaying the data to and fro. If one pony was too busy, a different pony would relay the data via a slightly different route. This critical feature is called redundancy.

This decentralized relay system results in many requests for the same data. These redundant requests threaten to overwhelm the Pony Express In the late 1990s, the Pony Express was replaced, or at least supplemented, by what at the time were called caching servers. With a caching server, your computer’s request for this page would not go all the way to Substack, but would instead go to a caching server. This reduces the burden on the Pony Express, and so it keeps the network from collapsing.

But a caching server becomes a critical node on the network. A network with a critical node is contrary to the original vision of the Internet. That vision was for a network that could work if any one node, or even a set of nodes, were taken out of commission. The original vision was to use redundancy to achieve reliability in the context of decentralization.

Another factor that creates critical nodes is usability. Even the most technically adept individual cannot master all of the computing skills needed to utilize the Internet’s information and communication capabilities. We use intermediaries, such as Google or Twitter.

Today’s so-called Internet is dominated by critical nodes. The paradigm of “cloud computing” is highly concentrated, not decentralized.

Major applications also create choke points. If the government wants to censor your tweets, it can go after whoever owns Twitter. If Elon does not comply, the government can go “up the stack” (or is it “down the stack”?) to a critical node or nodes in the overall network and shut down Twitter altogether.

Proponents of Bitcoin in particular and blockchain in general point out that blockchain enables decentralized intermediation. That means intermediation without critical nodes.

For example, property titles in the United States are located in filing systems in local jurisdictions, using primitive information storage—often just paper files. These county clerk offices are critical nodes, and they can function poorly. I recently received a tax lien notice for a house that we sold almost a year ago, because it can take the tax authority up to a year to update its ownership records.

The solution is a single definitive database of property records. In fact, a property records database, called MERS, was created years ago to support the mortgage securities industry. But that database is not legally definitive.

A definitive database could be a proprietary database, owned and operated by a corporation or a government authority. Or it could be a decentralized ledger, on blockchain. Setting up such a system would be costly. But the major impediment is not cost. It is that a definitive database would put the title-search and title-insurance firms out of business. They have sufficient lobbying power to maintain the status quo.

I want to emphasize that the engineered solution to the title mess could be either a centralized database or a blockchain application. But I am confident that a centralized version would make more efficient use of computer resources and be easier for an ordinary civilian to use.

Intermediaries and organizations that control critical nodes can be lazy and inefficient. That is the case with the governmental agencies and industry players that control the title process. But the inefficiencies in various sectors can be squeezed out without resorting to blockchain—provided we can remove the political or cultural impediments standing in the way.

The Internet produced revolutionary change by reducing the cost of exchanging digital information, making it feasible for software to “eat everything,” in Marc Andreessen’s words. Blockchain does not add to our capability to digitize—it only gives us a different method for doing so.

Blockchain applications are destined to be replaced by or supplemented with centralized systems, in order to enable them to scale up gracefully and be easy to use. If trading volume in crypto assets were to increase by orders of magnitude, a central clearing house would emerge to handle transactions in bulk. For masses of people to join a monetary ecosystem based on blockchain, intermediaries to provide digital wallets and digital currency conversion would have to emerge. These would create choke points and loci of control.

If applications developed on blockchain were to become more centralized, the alleged advantages of blockchain will fade. Intermediation will not disappear; instead, new intermediaries will emerge to make the applications user-friendly and scalable. You won’t be able to evade government control without exiting from major platforms; that will cut you off from the most widely-used applications.

The redundancy of the original Internet was designed for the military. If the military still wants that redundancy, then the solution is to take mission-critical applications off of the present-day Internet, with its critical nodes, and onto something like the original Internet.

For the rest of us, decentralized computing imposes too many costs to be the right solution. Redundancy is inefficient, and a lack of intermediaries creates usability barriers. I do not think that blockchain will take us back to the original Internet. Instead, I go back to the Internet Impossibility Theorem: achieving a decentralized society is a task for culture, not for engineering.