The Software Engineer with Alzheimer's

No, it's not me, although I am 71 years old. The challenge is getting Claude to remember what it was just doing.

Recall that this is a project to enable a professor to develop an immersive, online, text-based seminar. The professor provides the application with a detailed syllabus. The application takes it from there. It makes up a script for each week’s seminar for an AI professor and four AI students. A human user-student can take the seminar. During the seminar, the user’s comments and questions will be handled dynamically by the AI participants, so that the user will get more than just canned content.

Think of this as “substack for professors.” The professor can be prepared for the day when universities have been disintermediated and students learn directly from professors. Instead of trying to “scale up” by going through the arduous process of dealing with a textbook publisher and hoping for royalties, the professor can quickly deploy a seminar and charge subscription fees.

Anyway, that is not necessarily where I am headed. For now, this is a vanity project that will allow me to set up some of my own writing in immersive seminar format. And it will give me experience working with Claude.

• Claude will be responsible for drafting the scripts, based on material I supply and what we decide for each of the characters’ behavioral styles;

• Claude will, based on my proposals, develop code to make this work;

• during a seminar, Claude will respond to “API calls” to produce dialogue in real time in response to the human student’s comments and questions.

Working on this appeals to me because I can’t resist jumping into using a computer in a way that bumps up against some constraint. Fifteen years before Sam Altman was even born and twenty years before he moved to my (former) neighborhood, my high school had an avant-garde computer class that used a time-sharing system. I wanted to write a program that would bid bridge hands. Not just deal the cards—that would have been a reasonable thing to try. No, I wanted the computer to be able to sort the cards, count points, and choose the appropriate bids. Not just opening bids, either. The entire sequence.

The project met its demise when the paper tape broke. You typed in your BASIC code on a teletype terminal, and it punched holes in paper tape. When you were ready to try running it, you sent the paper tape through the terminal’s reader and via a modem (I think) to a remote mainframe computer. As I recall, I had a roll of paper tape about three inches thick containing my program when something happened and part of the paper tape tore, so that it could not be fed into the terminal.

I had something similar happen to me the other day, working with Claude. You know how some people are claiming that we will have Artificial General Intelligence real soon now? OK, tell me how you would characterize an employee who, after you just spent hours going back and forth about project descriptions, software configuration issues, passing useful files back and forth, and actual coding and debugging¹. You think you’re ready to begin the next step in the project, and suddenly the employee tells you “I’ve forgotten everything that we did the last several hours. Can we start all over?”

I don’t know what you would call such an employee, but I bet the word “general intelligence” is not what first comes to mind. What happened was that I wanted Claude to take in a lot of information, such as essays that I have written. Claude said that it would best remember it all if we kept extending our chat. We did that, and the chat grew longer and longer, until a message popped up saying that it had reached the maximum length. I would have to start a new chat. But a new chat for Claude has no connection to the old chat. Like an Alzheimer’s sufferer, Chat can remember just fine all the stuff it was trained on a long time ago. But it can’t remember anything recent.

I think I have found a way to work with Claude so that it does not forget important things. I will maintain 3 to 5 chats, each for a different purpose. I especially want to put in a separate chat our troubleshooting discussion, where we talk about bugs that need fixing and the like. This chat will capture a lot of back-and-forth that is not necessary to preserve, leaving room in the other chats to get longer without reaching the limit.

For the other chats, where we stay up to date on project goals, fine-tune the propensities of the characters, discuss software architecture², and develop basic seminar scripts, I will have Claude generate a summary of where we are at the end of the chat. I want to save each summary. It would be intuitively reasonable to be able to copy/paste that summary into a folder that Claude uses to hold the documents for the project. But no, you can’t. I will have to copy/paste the summary, save it to my computer, and then upload it to the folder. Otherwise, Claude’s right hand knoweth not what it’s left hand is doing.

Claude’s inability to keep track of what it is doing with you is not due to the ineptness of its designers. Keeping track, or “maintaining state,” is a software development challenge. It will be a challenge in the project I am attempting, even the “vanity project” version. Back in the 1990s, the Netscape browser team saw that maintaining state was going to be a useful capability. That is when the infamous “cookie” was invented.

Maintaining state is what is needed to create the vision of a “lifetime personal AI assistant,” who remembers everything about you. Getting from where we are now to the lifetime personal assistant may sound straightforward to some people. Altman is among those who seems to believe that it is achievable. I doubt that I will live to see it.

Another big hurdle for AI is sensory inputs. My project is strictly text-based Multimedia, with seminar participants seeing and hearing one another, may seem like an “adjacent possible.” Not really. It is actually quite difficult.

I like to say that a great way to put your foot in your mouth is to say that “AI will never be able to ____.” I will not use the word “never,” but I think that building an AI that can maintain state and handle rich multimedia inputs and outputs, on top of existing capabilities in AI, is something I would bet against happening soon.

I do believe that AI as it exists today will turn out to be very useful. I would bet even more strongly against those who scorn the chatbots and their like.

When I launched a commercial web site in April of 1994, I either read about or was told to my face by a lot of people that the web would never amount to anything.³ You don’t hear that said much anymore. Think about that when someone asserts that they will never have anything to do with LLMs.

I am going to get back to working with Claude on the vanity project. Now that I know that it has Alzheimer’s, I will be more careful about storing important project documents.

1. Claude’s code needed a few fixes, but most of the problems were on my end. I would save a file incorrectly, and Claude had to figure out what I’d done and tell me how to fix it. In my defense, there were a couple of times when Windows absolutely refused to let me save a file as anything other than .txt, and the only to fix it was to open up a command line and follow Claude’s instructions.

2. At one point during the now-useless long chat, I suggesting collecting some important components that will be shared across all seminar pages into separate files, so that we won’t have to put the same component on each page and then have to change it on each page when we want to make changes. Claude congratulated me for “thinking like a system architect,” and proceeded to “refactor” our work. Shouldn’t you be thinking like a system architect? I wondered.

3. I also pushed too hard on the capability of the software I was using, the Netscape web server. “Memory leaks” kept it running out of memory and crashing. We tried the new-and-untested JavaWebServer instead and got lucky.