"I have also observed that the adoption of formal bureaucratic processes in place of informal norms takes place around the point where a business crosses the Dunbar Number of about 150 employees."
Some observations:
Even when many things become officially formalized and bureaucratic, those de jure rules and procedures are a map that only bears slight resemblance to the actual territory. Actual practices and incentives deviate - sometimes quite substantially - from the picture one would get by taking the black letter rules at face value. New members of teams often need time to figure out the real deal. On the one hand, this state of affairs creates a lot of needed flexibility to bend and adapt in order to get things done without too many potential points of veto or having to make a federal case out of it. On the other hand, leaders can easily get so used to bending the rules that they take it for granted and become complacent about rubber-stamping whatever new official rules or policies come across their desk for approval. "Why scrutinize carefully? If it turns out to be a problem, we'll just bend informally like we always do." The predictable result is a premature and unwieldy accretion of often incoherent bureaucratization which - as soon as the culture of 'they are more like advisory guidelines, bend whenever you want' disappears with turnover or becomes otherwise untenable - traps the spider in its own web and gums up the works.
Where I work, the large organization does not have one character but two distinct ones.
On the one hand, there is a sub-Dunbar-sized inner circle and various cliques at or close to the executive / top staff officer level, and things in that circle often operate in a de facto sub-Dunbar manner. Discussions, decisions, and plans in that circle are extremely secretive and compartmentalized, with as many matters handled in a personal, relationship-based, confidential, face-to-face manner as possible. Whether their fellow insiders are worthy of it or not, things operate by necessity along the lines of "high trust equilibrium", that is, with mostly voluntary cooperation without mechanisms for monitoring, verification, or discipline. People out of this circle, even just below it, almost never have any good idea of what is being done or said or planned within it.
Outside the sub-Dunbar inner circle there is the vast super-Dunbar workforce. And, for any particular position in that workforce, one's experience of degree of formal regulation and bureaucratization seems to be correlated with one's overall distance in the hierarchy from the inner circle (in terms of status, rank, prestige, pay, skill-level, autonomy, decision-making power, etc.), the numbers of other people doing the same job, and the degree to which one is externally or internally-focused, that is, 'front-line', interacting with people and objects in the real world outside one's own organization, instead of processing, analyzing, and manipulating abstract ideas, numbers, words, and symbols.
Still, even though it's a spectrum with many levels and flavors of effective bureaucratization, there is a uniquely clear and stark discontinuity between the inner circle and everything else.
One interesting aspect of this dual-charactered organization, at least where I work, is that there is a severe amount of friction and impairment of communication, direction, guidance, and trust at the border between the two worlds, which in turn generates a lot of angst, drama, and general dysfunction. My impression is that elite managers still have not quite discovered how to lead, manage, and administer effectively in such organizationally discontinuous circumstances, or else, it simply takes a combination of great skill, experience, a highly beneficial incentive structure, and independence to avoid these issues.
One thing to note is that the example of finance traders and small teams is just one case where regulators, either within a form or outside it, contribute to the effective organization size. Payment card standards, safety for healthcare or avionics or autonomous vehicles, government security rules -- all of these impose hierarchies and separation of roles that might be theoretically suboptimal for a given project, but
The trouble is that most off the shelf ERP solutions need a LOT of customization to your business. Not just the product itself, but the way the process flow through the business works, who enters what, when, etc. There is a whole industry of consultants to get e.g. SAP working right for your company after you paid for it and had it implemented the first (or second, or third...) time to fix the problems.
To pick on SAP since I did a lot of work with it a while back, the system is not designed by people who ever intend to use it themselves, was sold to people who don't have any intention of using it themselves (that is other people's job), and tends towards "Here's 20,000 switches. You will use 20, but will probably need to use 25, and those won't necessarily overlap. Oh, and God help you if the other 19,975 switches are not flipped appropriately. Also, there isn't a manual less than 5 inches thick." Even the user interfaces are awkward and not user friendly, even by the standards of spreadsheet or statistical analysis programs.
The other big problem is that you have to model your process in your ERP system for interesting things like forecasting or production planning. For some businesses that is really easy, but for others it can be a nightmare. For some reason, process modeling is difficult for humans. (There seem to be things you can do growing up to get better at it, but in my experience the vast majority of people really struggle with it, on both the IT and operations side of the business.)
Where I work, we have in-house developers who are skilled at creating a wide variety of Microsoft Enterprise-based systems, CRM website-accessible databases with PowerBI visualization, and so forth. These projects are almost all of the nature of internal adminsitration, not really for managing relationships and accounts with vendors and customers.
For a simple project, it's effectively free as your sub-organization only pays a negligible nominal price for it, and even for a complicated project it's intentionally underpriced and bears no real relationship to the true costs.
An Econ-101 analysis would predict that these services would either be heavily over-used, or rationed by long waiting lists, or else require a huge budget to satisfy demand, or else people were generally content and satisfied with the IT / automation systems and processes they had, to the point that they didn't think it worthwhile to endure the costs of learning and implementing something new, or perhaps the developers had a reputation for producing low-quality, buggy junk that didn't match one's specifications, such that there was really no point in putting in a project development request.
But actually, and interestingly, none of those things are true. The size and budget for the development office is fairly small and reasonable, there is practically no rationing or waiting, they have a well-deserved reputation for high-quality, and people do want new, better systems and complain all the time about real and obvious problems with the current systems.
So, what's the deal? The deal is that the things that are 'broken' or 'inadequate' with the current systems are not really IT problems and thus can't be solved with IT solutions, but that the real problems are either unmentionable or irremediable issues with leadership.
That is, executives often complain about things that are the indirect consequence of their own decisions or style, but when subordinates are asked to 'do something' about it, 'come up with a system to prevent this from happening again', everyone just says, 'yes, of course, we'll get right on it' instead of pushing back with uncomfortable truths about cause and effect and trade-offs.
Which feeds and perpetuates an endless and amusingly absurd tragi-comic cycle of redevelopment. I don't know if Scott Adams ever explored the theme in Dilbert, but from talking to a lot of people who have been at the right levels for long enough, everyone seems to have had the same experience and observation of how it tends to go down.
(A) System 1
(B) Steady drip of problems
(C) Frustration gradually builds, executives are annoyed and demoralized by having to deal with such a 'broken' organization where everyone just accepts or is forced to tolerate a high level of constant brokenness, never, ever thing it might be due to their own behaviors.
(D) One or several big fiascoes
(E) Boiling point reached, "Unacceptable! Do something!"
(F) "Yes sir!"
(G) System 2
(H) Repeat (B)-(F)
(I) System 3
(J) Repeat (B)-(F)
(K) System 4
(L) Repeat (B)-(F)
(M) System 1
Fundamentally, IT systems which require human complements to perform their function also require whatever organization capital is necessary to make sure those human are doing whatever they are supposed to be doing to make those IT system work as intended. If you can't do that, and can't completely substitute for those humans but totally automating their tasks, (and often, wouldn't want to even if one could), then you can never develop your way out of the core issue. But you *can* keep pretending that you can, and just go in endless loops around the above cycle.
Yea, that rings pretty much 100% true to me. Sometimes the problem stems from the system not doing what people need, and sometimes from people not actually using the system. In both cases it is a leadership problem, either because they set up a process and didn't match the system to it, or people just don't know the process they are supposed to be doing for a variety of reasons. Or the leadership just wants bad things.
This gets back to the problem of there often being no effective way in large firms organized and managed in currently typical ways for leaders to learn about or for subordinates to communicate ugly truths or bad news accurately and all the way up the chain of command.
In any big org there are lots of strong incentives working against the creation of such accurate awareness, which is likely why it seems to be so very common a problem.
I suspect that it's such a hard issue that it imposes a kind of selection pressure of which activities end up succeeding in the overall marketplace, and I think the Michael Porter / Pieter Thiel insight would be that these would evolve in the direction of favoring activities which do not depend for their success on executives being able to effectively pull in and rationally respond to negative signs and feedback. Cough, Microsoft, cough. If you needed to do it well to be successful, and it's too hard to do well, then you won't be successful.
The really scary implication is that there is really nothing one can do about it*, and that as soon as things start to fall apart in this direction and rot really sets in, then it is just impossible to reverse without starting from scratch. So, when you see institutions show severe pathologies along these lines, it's time for major surgery, demolition and renovation.
*Not 'nothing', there are some ideas out there, but they would require radical steps, they are not in the character of usual reforms or minor tweaks, and they are likely to get strong resistance as tending to expose some natural and perhaps necessary hypocrisies in ways too stark to tolerate.
Hanson has proposed organizationally internal prediction markets, which is one way for executives to learn what their own workers really think about the likely success or failure of certain initiatives. So far, no takers, it may be too incompatible with the necessary foundations for human leadership.
Another idea could be a kind of "undercover boss" situation of internal espionage in which executives intentionally plant agents into the body of the organization and incentivizes them to quietly blow the whistle when what they see and hear deviates substantially from official reporting. Beyond mere substitution via automation with sensors and algorithms, this seems infeasible.
Agreed. I think fundamentally there is a limited lifespan for organizations, and size tends to lower this lifespan, all else being equal. My suspicion is that most of government regulation that benefits large businesses by protecting them from competition tends to extend this lifespan. The most dramatic example probably being GM, which should have died a few different times.
Government organizations have the problem of never dying, becoming ever more cancerous and broken, eventually ending as shambling ghouls that harm and infect everything around them. It is hard to imagine a government COO that could put them down, considering that neither congress nor any president seems willing or able to do so.
I'd agree with that, probably 90%. There are probably some companies that can do it better, but it depends a lot on getting lucky with having the right talent on hand. Using off the shelf systems also require having a lot of talent, but probably with a lower threshold.
It’s “The Church of Jesus Christ of Latter-day Saints.” You left out the important part.
"I have also observed that the adoption of formal bureaucratic processes in place of informal norms takes place around the point where a business crosses the Dunbar Number of about 150 employees."
Some observations:
Even when many things become officially formalized and bureaucratic, those de jure rules and procedures are a map that only bears slight resemblance to the actual territory. Actual practices and incentives deviate - sometimes quite substantially - from the picture one would get by taking the black letter rules at face value. New members of teams often need time to figure out the real deal. On the one hand, this state of affairs creates a lot of needed flexibility to bend and adapt in order to get things done without too many potential points of veto or having to make a federal case out of it. On the other hand, leaders can easily get so used to bending the rules that they take it for granted and become complacent about rubber-stamping whatever new official rules or policies come across their desk for approval. "Why scrutinize carefully? If it turns out to be a problem, we'll just bend informally like we always do." The predictable result is a premature and unwieldy accretion of often incoherent bureaucratization which - as soon as the culture of 'they are more like advisory guidelines, bend whenever you want' disappears with turnover or becomes otherwise untenable - traps the spider in its own web and gums up the works.
Where I work, the large organization does not have one character but two distinct ones.
On the one hand, there is a sub-Dunbar-sized inner circle and various cliques at or close to the executive / top staff officer level, and things in that circle often operate in a de facto sub-Dunbar manner. Discussions, decisions, and plans in that circle are extremely secretive and compartmentalized, with as many matters handled in a personal, relationship-based, confidential, face-to-face manner as possible. Whether their fellow insiders are worthy of it or not, things operate by necessity along the lines of "high trust equilibrium", that is, with mostly voluntary cooperation without mechanisms for monitoring, verification, or discipline. People out of this circle, even just below it, almost never have any good idea of what is being done or said or planned within it.
Outside the sub-Dunbar inner circle there is the vast super-Dunbar workforce. And, for any particular position in that workforce, one's experience of degree of formal regulation and bureaucratization seems to be correlated with one's overall distance in the hierarchy from the inner circle (in terms of status, rank, prestige, pay, skill-level, autonomy, decision-making power, etc.), the numbers of other people doing the same job, and the degree to which one is externally or internally-focused, that is, 'front-line', interacting with people and objects in the real world outside one's own organization, instead of processing, analyzing, and manipulating abstract ideas, numbers, words, and symbols.
Still, even though it's a spectrum with many levels and flavors of effective bureaucratization, there is a uniquely clear and stark discontinuity between the inner circle and everything else.
One interesting aspect of this dual-charactered organization, at least where I work, is that there is a severe amount of friction and impairment of communication, direction, guidance, and trust at the border between the two worlds, which in turn generates a lot of angst, drama, and general dysfunction. My impression is that elite managers still have not quite discovered how to lead, manage, and administer effectively in such organizationally discontinuous circumstances, or else, it simply takes a combination of great skill, experience, a highly beneficial incentive structure, and independence to avoid these issues.
“In contrast, Amazon tries to keep its software projects small enough so that they can be operated by two-pizza teams.”
What a clever heuristic. Also brings a whole new meaning to the phrase “one-man team”
One thing to note is that the example of finance traders and small teams is just one case where regulators, either within a form or outside it, contribute to the effective organization size. Payment card standards, safety for healthcare or avionics or autonomous vehicles, government security rules -- all of these impose hierarchies and separation of roles that might be theoretically suboptimal for a given project, but
... but mitigate the risk to the stakeholders who the regulators represent.
The trouble is that most off the shelf ERP solutions need a LOT of customization to your business. Not just the product itself, but the way the process flow through the business works, who enters what, when, etc. There is a whole industry of consultants to get e.g. SAP working right for your company after you paid for it and had it implemented the first (or second, or third...) time to fix the problems.
To pick on SAP since I did a lot of work with it a while back, the system is not designed by people who ever intend to use it themselves, was sold to people who don't have any intention of using it themselves (that is other people's job), and tends towards "Here's 20,000 switches. You will use 20, but will probably need to use 25, and those won't necessarily overlap. Oh, and God help you if the other 19,975 switches are not flipped appropriately. Also, there isn't a manual less than 5 inches thick." Even the user interfaces are awkward and not user friendly, even by the standards of spreadsheet or statistical analysis programs.
The other big problem is that you have to model your process in your ERP system for interesting things like forecasting or production planning. For some businesses that is really easy, but for others it can be a nightmare. For some reason, process modeling is difficult for humans. (There seem to be things you can do growing up to get better at it, but in my experience the vast majority of people really struggle with it, on both the IT and operations side of the business.)
Where I work, we have in-house developers who are skilled at creating a wide variety of Microsoft Enterprise-based systems, CRM website-accessible databases with PowerBI visualization, and so forth. These projects are almost all of the nature of internal adminsitration, not really for managing relationships and accounts with vendors and customers.
For a simple project, it's effectively free as your sub-organization only pays a negligible nominal price for it, and even for a complicated project it's intentionally underpriced and bears no real relationship to the true costs.
An Econ-101 analysis would predict that these services would either be heavily over-used, or rationed by long waiting lists, or else require a huge budget to satisfy demand, or else people were generally content and satisfied with the IT / automation systems and processes they had, to the point that they didn't think it worthwhile to endure the costs of learning and implementing something new, or perhaps the developers had a reputation for producing low-quality, buggy junk that didn't match one's specifications, such that there was really no point in putting in a project development request.
But actually, and interestingly, none of those things are true. The size and budget for the development office is fairly small and reasonable, there is practically no rationing or waiting, they have a well-deserved reputation for high-quality, and people do want new, better systems and complain all the time about real and obvious problems with the current systems.
So, what's the deal? The deal is that the things that are 'broken' or 'inadequate' with the current systems are not really IT problems and thus can't be solved with IT solutions, but that the real problems are either unmentionable or irremediable issues with leadership.
That is, executives often complain about things that are the indirect consequence of their own decisions or style, but when subordinates are asked to 'do something' about it, 'come up with a system to prevent this from happening again', everyone just says, 'yes, of course, we'll get right on it' instead of pushing back with uncomfortable truths about cause and effect and trade-offs.
Which feeds and perpetuates an endless and amusingly absurd tragi-comic cycle of redevelopment. I don't know if Scott Adams ever explored the theme in Dilbert, but from talking to a lot of people who have been at the right levels for long enough, everyone seems to have had the same experience and observation of how it tends to go down.
(A) System 1
(B) Steady drip of problems
(C) Frustration gradually builds, executives are annoyed and demoralized by having to deal with such a 'broken' organization where everyone just accepts or is forced to tolerate a high level of constant brokenness, never, ever thing it might be due to their own behaviors.
(D) One or several big fiascoes
(E) Boiling point reached, "Unacceptable! Do something!"
(F) "Yes sir!"
(G) System 2
(H) Repeat (B)-(F)
(I) System 3
(J) Repeat (B)-(F)
(K) System 4
(L) Repeat (B)-(F)
(M) System 1
Fundamentally, IT systems which require human complements to perform their function also require whatever organization capital is necessary to make sure those human are doing whatever they are supposed to be doing to make those IT system work as intended. If you can't do that, and can't completely substitute for those humans but totally automating their tasks, (and often, wouldn't want to even if one could), then you can never develop your way out of the core issue. But you *can* keep pretending that you can, and just go in endless loops around the above cycle.
Yea, that rings pretty much 100% true to me. Sometimes the problem stems from the system not doing what people need, and sometimes from people not actually using the system. In both cases it is a leadership problem, either because they set up a process and didn't match the system to it, or people just don't know the process they are supposed to be doing for a variety of reasons. Or the leadership just wants bad things.
This gets back to the problem of there often being no effective way in large firms organized and managed in currently typical ways for leaders to learn about or for subordinates to communicate ugly truths or bad news accurately and all the way up the chain of command.
In any big org there are lots of strong incentives working against the creation of such accurate awareness, which is likely why it seems to be so very common a problem.
I suspect that it's such a hard issue that it imposes a kind of selection pressure of which activities end up succeeding in the overall marketplace, and I think the Michael Porter / Pieter Thiel insight would be that these would evolve in the direction of favoring activities which do not depend for their success on executives being able to effectively pull in and rationally respond to negative signs and feedback. Cough, Microsoft, cough. If you needed to do it well to be successful, and it's too hard to do well, then you won't be successful.
The really scary implication is that there is really nothing one can do about it*, and that as soon as things start to fall apart in this direction and rot really sets in, then it is just impossible to reverse without starting from scratch. So, when you see institutions show severe pathologies along these lines, it's time for major surgery, demolition and renovation.
*Not 'nothing', there are some ideas out there, but they would require radical steps, they are not in the character of usual reforms or minor tweaks, and they are likely to get strong resistance as tending to expose some natural and perhaps necessary hypocrisies in ways too stark to tolerate.
Hanson has proposed organizationally internal prediction markets, which is one way for executives to learn what their own workers really think about the likely success or failure of certain initiatives. So far, no takers, it may be too incompatible with the necessary foundations for human leadership.
Another idea could be a kind of "undercover boss" situation of internal espionage in which executives intentionally plant agents into the body of the organization and incentivizes them to quietly blow the whistle when what they see and hear deviates substantially from official reporting. Beyond mere substitution via automation with sensors and algorithms, this seems infeasible.
Agreed. I think fundamentally there is a limited lifespan for organizations, and size tends to lower this lifespan, all else being equal. My suspicion is that most of government regulation that benefits large businesses by protecting them from competition tends to extend this lifespan. The most dramatic example probably being GM, which should have died a few different times.
Government organizations have the problem of never dying, becoming ever more cancerous and broken, eventually ending as shambling ghouls that harm and infect everything around them. It is hard to imagine a government COO that could put them down, considering that neither congress nor any president seems willing or able to do so.
I'd agree with that, probably 90%. There are probably some companies that can do it better, but it depends a lot on getting lucky with having the right talent on hand. Using off the shelf systems also require having a lot of talent, but probably with a lower threshold.