I posed the question of whether institutions fail for external reasons (changes in the environment) or internal reasons (natural decay of some sort).
I received a preview copy of a book by Jonathan Heskel and Stian Westlake with the vague title of Restarting the Future, which gives an example of an external factor. They argue that an important change in the environment is an increase in the importance of intangible assets. This puts pressure on a number of institutions. For example, patents take on more significance, so that mistakes in the design or execution of patent law are more serious. Debt finance works tolerably well when there are tangible assets as collateral, but not so well for intangible assets.
An example of an internal dynamic is the drive for expansion and power. An organization creates a department to solve a problem, and the leader of that department wants to increase its size and scope. You end up with an unwieldy bureaucracy.
Sometimes, institutions react to threats in ways that make things worse. For example, Ross Douthat sees the Catholic Church in the wake of the sexual revolution having a hard time recruiting priests, which made the Church reluctant to aggressively police the behavior of those priests that it had.
Another example is the Savings and Loan industry. In the 1960s, they funded long-term fixed-rate mortgages with short-term deposits. In the 1970s, this made them unprofitable. They responded by getting into riskier loans and ended up losing more money.
Also, when an institution faces a troubled future, the executives have an incentive to loot the remaining assets by awarding themselves high salaries and bonuses. This happened when the Berlin Wall fell and Eastern European companies were privatized.
In general, businesses are forced to adapt to external threats or else they disappear. Government institutions are less adaptive.
Consider schools during the pandemic. Private schools were more likely to find a way to accommodate parents, both the parents who wanted in-person schools and those who did not. But public schools gave in to the minority of parents who did not want in-person schools as well as to the constituents in the teachers’ unions who wanted schools closed. They shut down schools even though it cost them a tremendous amount of good will with parents.
How do institutions get taken over by intransigent minorities? Everyone else is looking for consensus, so if one group is intransigent the only way to have consensus is to go along with them.
Some common external factors: new technology, war, population movements, cultural changes. One example is the sorting of our work force into college-educated and non-college educated, along with assortive mating.
Internal factors include careerism, gamable education credentials, elevating DEI over merit.
The transition from a founder may put the company in hands of a political power-grabber rather than a creative, forward-looking leader. For example, a social media founder might have an ideal of free speech, but the successor may be more interested in appeasing those who want to censor.
One simple narrative for the U.S. in recent decades is that during the Vietnam War and its aftermath, radicals became professors and gradually took over universities, which in turn radicalized students who then moved in journalism and corporations. But is this a reliable story? Are college students so susceptible to radical teaching?
What to do about lack of integrity, e.g. Enron? The Board fails in that case, so you get Sarbanes-Oxley. Does that legislation solve the problem? Can you legislate morality in that way? Activision shows that ESG investing does not necessarily ensure that companies follow the ethical standards. Bebchuk’s research says that managers are incented to focus on profits regardless. For other goals, they can make symbolic gestures.
What about counter-examples of institutional improvement? Articles of Confederation replaced by Constitution; China under Deng; Jimmy Carter appointing Alfred Kahn and Paul Volcker.
How might journalism rebound? A more distributed model, with truth-seeking journalists outside of the legacy media getting more attention
How about religion? Perhaps religious Catholics will rely less on the structure of the church. Similarly with other religions.
Perhaps the Internet is creating uneven distributions of many cultural traits. Perhaps there will be various religious cults, a cult of extreme truth-seekers, or what have you.
In Los Angeles, private schools have been no refuge for parents or students. My son attended a private school and we have friends whose children attended various others; together, these families attend the "best" private schools in Los Angeles, and they all closed down for over a year of remote learning. They have all re-opened this year, but all of them continue to enforce all day masking with few exceptions. Additionally, they have all embraced Critical Race Pedagogy. Our school has Zoom lectures from both Kendi & D'Angelo & Kendi has been re-invited despite poor attendance at his last appearance. They have created & continue to promote racially segregated monthly affinity group virtual meetings, again, poorly attended by all reports.
Seventeen of my son's class of 68 chose not to return this year (including us). Some left the state, a few of us are home schooling. But private education, at least in this deep Blue city, is not the answer.
Where I work, one kind of failure is an entrenched 'tragedy of the commons'.
To put it very generally to protect the innocent (i.e., me), many key executives and staff face a common, individual choice between a hard right and an easy wrong.
The hard right is slow, cumbersome, annoying, makes one reliant on third parties and multiple layers of middlemen, and requires tremendous reserves of patience while waiting in 'receive mode' for other people to do their jobs. It is overly inflexible because there is no good way to 'appeal' for some kind of exception justified by what one wants to argue is an exigent circumstance, because of the cry-wolf problem, in which everybody says their pet project is exigent. The hard right way, however, is the only way one even gets close to preventing a total collapse into disorganized chaos.
The easy wrong is just going around all that, doing things by oneself, "asking forgiveness instead of permission", cutting out all the middlemen, and dealing directly and privately with someone hierarchically-distant. This is super efficient if one person does it one time. If everybody does it all the time, everything breaks down and productivity collapses as people duplicate effort and work at cross purposes and generally fail to stay on the same sheet of music.
So, the incentive for individuals is always to want everyone else to follow the norm, but to have the option to violate the norm on their own initiative when it's their own project.
What happens is that an institution may start with everyone following the norm, but gradually people make exceptions for themselves, other people see that happen, start to do it too, and eventually you have a culture of circumvention and general disrespect of 'official' norms in general, as being something only naive chumps would follow voluntarily, or low status workers without autonomy would be compelled to follow involuntarily. "Taxes are for the little people."
Once established it is extremely hard to reverse. For one thing, a lot of this operates on the honor system, and few have the stomach to use monitoring and surveillance tools to detect violations and then to hold people accountable for them using consequences of whatever severity are necessary to deal with the problem. High status executives and staff would revolt over such a perceived hit to their autonomy.
Enforcement of anything in general takes personalities with real authority and top cover and who are not at all averse to direct personal confrontation, which is alien to the culture at the executive level in many institutions. Also, top staff know where the bodies are buried, and ruffling feathers too much in the corporate soap opera is a good recipe for them leaving, leaking, or otherwise undermining leaders and the mission.
Leaders at the elite level want maximum flexibility for themselves, and so want to be able to make exactly this kind of exception. They want to be able to talk and direct people with secrecy, and not worry about what happens to people who needed to know but now don't. They don't want to be accused of hypocrisy, or being biased or unfair, "you made an exception for her, but not for him?". It is too difficult if not impossible to articulate any clear and well-defined 'law' about the matter. Sometimes top leaders *want* certain subordinates to do whatever it takes - i.e., violate the official norms - to expedite and shake something loose, but don't want *other* subordinates to know that leaders can and do authorize exceptions to norms advertised as "always and for everyone", or that the fallout they have to deal with was a consequence of that leader's decision.
Once this kind of rot has set in, there is really nothing else one can do but a hard reset which like with software often means starting over from scratch in many areas with a completely new set of people and rules, which at least buys you some time before organizational weeds and social entropy accumulate and need to be cut back and reset again. It’s like regular maintenance, and if institutions keep deferring it, they eventually break catastrophically.