Software Development in the Old Days
Try being agile on an IBM 370
[Note: On Tuesday, November 28, at 3 PM New York time, Rebecca Burgess and I will be discussing Emmanuel Todd’s Lineages of the Feminine, which I wrote about here. No charge, but you must register.]
Andrew McAfee, in The Geek Way, highlights the advantages of developing software using rapid evolution rather than extensive planning. You achieve better results more quickly. I should note that there is an analogy to the advantage of using markets rather than government.
But McAfee makes it sound as if software engineers and managers discovered agile software development because they are just smarter today than they were in the 20th century. That is not how I interpret the history.
My guess is that there are relatively few people who can tell you what both of these acronyms mean in the context of software development:
JCL
MVP
JCL stands for “job control language,” a relic from the IBM mainframe era. MVP, which I associate with “most valuable player,” now in the world of software businesses stands for “minimum viable product.” In the agile approach, you try to get to an MVP as quickly as possible and then iterate from there.
My point is that in the world of JCL, which probably started around 1964 when the IBM 360 mainframe was introduced, it would have been much more difficult to get to an MVP and then evolve forward. Mainframe computing did not allow for sufficiently rapid feedback.
In 1975, I was a research assistant for the Congressional Budget Office, trying to run a large policy-analysis model on an IBM 370 mainframe, which was controlled by a subcommittee of the House Administration Committee. The model and its data were stored on a large tape. To run a program, I had to make punch cards and walk several blocks between CBO and a House office building, hand the punch cards to a staffer, and then come back later in the day to get the output. You could make one attempt per day, maybe two. The trickiest part was using the correct Job Control Language (JCL) to instruct the computer operator to mount the tape and to perform other crucial functions. It took me several days just to get the JCL right, so that I could start to work on the program itself.
Today, using the Internet, a developer can get instant feedback on an attempt to run a computer program. You could try dozens of times a day—not that I would recommend that cadence.
If you are only going to get feedback once or twice a day on a computer program, then it pays to undertake a lot of planning before you begin coding. For complex corporate systems, this meant involving a lot of people attending meetings, articulating needs, and creating system diagrams. McAfee derides this process for being cumbersome and slow. Indeed, it is so slow that by the time a project is mostly done, design flaws and new requirements have been discovered. What seemed like the last 10 percent of the project at the outset now takes as much or more time as the first 90 percent. This makes projects likely to come in late and over budget, if they are not abandoned altogether.
Today’s agile software development works much better. You work quickly to get an MVP. The issues that would show up late in the 20th-century process instead are surfaced right away. You then work quickly to correct bugs and add new features.
The point I wish to emphasize is that you simply could not execute this way in the IBM mainframe era. The IBM mainframe did not offer the software engineer feedback at anywhere near the speed that is needed for agile programming. In my research assistant days, a computer operator was a major bottleneck for all of us trying to use the mainframe. The operator had to mount tapes and change other settings before a program would even start to run.
The infamous problems with building the Obamacare web site were due, I suspect, to the fact that it was supposed to interface with existing systems, including at the Internal Revenue Service, that were running on 20th-century mainframes. If a system is on an Internet server, with the proper permission you can just “call” it using an Application Programming Interface. Systems on IBM mainframes were never built with APIs that enabled you to call them in real time.
Trying to solve the problem of developing a complex system using an organized planning process is always a challenge. It is better to use modern methods of evolving a system through an iterative process of trial and error. (It is also easier nowadays because many pre-packaged software components are available.) McAfee can cite examples of organizations that still tried to develop software in the 21st century using 20th century methods, and for them I have no sympathy.
But the software engineers of the 20th century should not be faulted for failing to invent the agile approach. They did not have the rapid feedback available to 21st-century developers on the Internet. They could not have used agile methods even if they had wanted to.
I just went thought an "agile to mvp then refine" process and it was miserable and resulted in the worst product I've used in a decade, and believe me, that's a high bar to have surpassed.
To be fair, my organization is also increasingly dysfunctional, and so there is an extension of Conway's Law that the health of the software development process also resembles the health of the organization's core functioning, and furthermore that there is no one right approach to development, but instead, very different approaches are needed for different organizations or things will end up in disaster.
"Agile MVP" seems better suited to organizations with strong leadership, hierarchies, accountability, (I.e., some one person is known to everyone to be in charge of the project, and is empowered to actually be in charge), well-defined responsibilities and processes, and experience at formulating and communicating vision and requirements precisely and assessing objectively.
None of that is how you would describe my organization, so the approach crashed on the rocks. After MVP, there was zero motive all around to make even the biggest bang-for-buck improvements, so everything froze with all but severe work stoppage bugs going on the indefinite backlog.
You can imagine what the minimally viable version of anything you own would be like, the version that could have been developed with the absolute smallest budget in the minimum amount of time and still technically satisfying some core function, and then the prospect of being stuck forever with that version of it.
There is a similar tension in hardware engineering. Take for example the case of building an optical assembly such as a microscope or telescope that requires mechanical, electrical, software, and optical engineers.
There are certain optical assemblies in which no MVP can be built because without a custom-designed and custom-fabricated lens, no feedback can be obtained from the system. Designing and manufacturing the lens assembly whether it be reflective or refractive is almost always the longest lead item.
This puts tremendous pressure on the optical engineers. And to add more pressure, managers typically don’t understand enough optics to understand why it would take so long to design and fabricate a lens assembly, so the optical engineers have to spend extra time educating project managers and executives of nuances such as lens fabrication tolerancing in which the optical designer iteratively goes back and forth with the fabricate to learn which tolerances are too tight. This often requires changing the surface shape of the lens, which requires full re-optimization of all the lens dimensions and the mechanical mounting scheme.
Fabricators typically are unwilling to give away their secret techniques nor their weaknesses, so typical lens fabrication tolerances that are published don’t always apply. For example, some lens materials are so unique there’s nothing published to use as a reliable reference.
On top of this, there is a subset of optics called straylight engineering that is very important for seeing faint objects in a noisy background. For example camouflaged projectiles coming your way, or trace molecules that signal deadly cancer in one’s blood. In these life or death cases the systems sensitivity must meet specification, otherwise the missiles or cancer will not be detected in time and people will die.
If people only knew how much concentration it requires to solve such complex problems and how much it would benefit humanity to be able to detect cancers months sooner...
So people, read The Three Languages of Politics, aim for a civil dialogue, end the cancellation attempts, so your optical engineers can design better lenses, so that we can increase life expectancy and the quality of life.
This message has been brought to you by former optical engineers of Silicon Valley that no longer practice optics because well who can design lenses in the face of lockdowns, school closures, Supreme Court nominee cancellation attempts, gender transition discussions at public schools, so on and so forth.
You won’t have nice optics until you read Arnold’s book.