We're All Wrong About AI
Remember when we thought personal computers were for tracking recipes?
Looking back at the Apple II, in 2018 Michael Halvorson recalled marketing blurbs
stating that you can use the device to teach your kids arithmetic and make learning fun, manage household finances, chart the stock market, track your recipes and record collection, and control your home.
I don’t have to spell out the applications for personal computers that have emerged since then.
I think that with the latest developments in AI, we are at the “track your recipes” stage of imagination. What people think that AI will do is probably poorly correlated with how AI’s actually will be used. Meanwhile, I have a few pet peeves about claims people are making concerning what AI’s will and won’t do.
AI’s Can be Creative
Anyone who makes the argument that AI’s can never be creative, and that you need humans for that, probably understands neither human creativity nor the working of the current generation of AI.
Human creativity does not mean coming up with ideas full-blown out of nowhere. Creativity is recombinant. A creative idea takes two or more previous ideas and combines them in a way that they have never been combined before.
Current versions of AI represent words or other concepts (such as musical sounds) as multidimensional vectors. All it takes for an AI to be creative is for it to try to add vectors that have never been added before.
Adding two arbitrary vectors is very unlikely to result in a new concept that is at all useful. For an AI to be creative, it will have to be able to extract from many possible vector combinations those rare syntheses that are worth retaining.
Human creativity is no different. Pop songwriters have no problem coming up with different possible chord progressions. The challenge is to choose the ones that are both novel and pleasing.
A schizophrenic and I can both come up with novel word sequences. The schizophrenic is better at generating them. I am better at filtering out the ones that don’t provide meaning.
If you wanted it to, a large language model could beat the most prolific schizophrenic at generating novel content. But in order to be appreciated as creative, AI’s will have to become excellent at filtering out the novel content that doesn’t work. Once they learn to do that, AI’s will reach human or super-human levels of creativity. If there is some barrier to AI’s becoming excellent at such filtering, I do not see it.
AI is not Google on steroids
When I was searching for a reference to the early days when Apple was touting home computers as devices to help track recipes, I tried several different large language models. None of them helped me. I found the quote that I used for this essay using plain old Google.
Large language models are trained using data on the Internet. But unlike Google, they are not trying to analyze the data on the Internet. Google is restructuring and modeling data on the Internet in order to feed an algorithm for search. Large language models are using the data in order to learn how to write (or draw, or make sounds).
AI can understand you as an individual
Many people are focused on generic AI skills. They see it drafting news articles. They see it doing preliminary data analysis. They see it writing software. It could do all of these things without knowing anything about the person it is doing it for.
But another possibility captivates many of us. That is, an AI will learn about you as an individual, becoming your companion. For a child, it becomes a tutor. For an adult, it becomes a personal assistant. For anyone, it becomes a friend.
The potential use of an AI as a friend immediately calls to mind the “AI girlfriend” or romantic partner. I am dubious about this, but entrepreneurs and some consumers are trying it.
An AI relationship does not have to involve romance in order for it to be personal. An AI just has to remember past interactions with you and use that memory to better interact with you and align itself to your preferences.
Returning to where this essay began, I do not believe that we know enough to predict with any accuracy which uses of AI will take off and which will prove to be either too trivial (recipe tracking) or too elusive (maybe it’s too much to hope that AI will allow children to escape from school-prison). The best way to find out is to work with it, not to dismiss it.
I will play Devil's Advocate. Arnold wrote:
"A schizophrenic and I can both come up with novel word sequences. The schizophrenic is better at generating them. I am better at filtering out the ones that don’t provide meaning."
What is it in you that filters out nonsense from meaning? Would an AI randomly combine words, come up with Lewis Carroll's "Jabberwocky" or Wallace Steven's "The Snowman", and not reject both as nonsense? Right now, it is humans that are doing the real filtering out of what AI produces, and to my way of thinking, humans will still be doing the filtering out 20 years from now by writing the rules for what the AI outputs for consumption.
The comment about creativity strikes me as naive, for two reasons: (1) it does not match history. What mashup of previously existing ideas is represented by special relativity, Joule's discovery of the mechanical equivalent of heat, the valence-bond theory of the chemical bond, integral calculus, or the Fourier transform? History is strewn with ideas that came absolutely from the blue, that cannot possibly be described as some mash-up or recombination fo existing ideas. (I mean, *maybe* it has some validity in the area of punditry and fiction writing, where I kind of am inclined to agree there isn't much new under the Sun since Cato the Elder thundered in the Senate about Carthage, but it would be unwise to generalize from this very limited area of human invention to all the areas where we have had, and continue to have, genuinely new ideas.)
(2) If it were true that it were possible to construct sense by filitering nonsense, then LLMs would *already* work that way. It's not hard to create a Perl script that generates endless numbers of random sentences (even grammatical sentences). Surely the easiest way to construct an AI would therefore be to hook such a program to a "filter" that "merely" discards all the nonsense sentences and keeps the good ones? Why wasn't that done in, say, 1975?
But that of course is *not* how LLMs work, and such an approach would be sterile -- because the space of nonsense is *so* much larger, in practice effectively infinitely larger, that even with a guaranteed perfect filter in place (i.e. glossing over the considerably challenge in even defining, let alone building, a filter that reliably sorts sense from nonsense), you simply get lost in the space of nonsense without ever stumbling upon sense. The old aphorism about hoping that 10 million monkeys randomly typing on a typewriter would reproduce Shakespear, and "all" you need to do is have a function to recognize it when they do, comes to mind. There have been similar arguments about protein folding and more speculatively about evolution, but this model in general is naive and doesn't work in practice. I suggest that's *why* LLMs had to approach the problem from entirely the other direction: "begin* with an enormous corpus of sense, generated by human beings, and attempt to detect patterns in it, and then construct other sentences using those discovered patterns. That bypasses the otherwise insurmountable problem of wandering around in an infinite sea of meaninglessness and hoping to stumble on a tiny islet of meaning by accident.
In fact, I suggest we don't know why and how humans are creative. The recombination idea seems naive to me, not to mention out-dated -- it has not succeeded in any area in which it has been applied. So I'm very doubtful we will learn anything about creativity by just making an even larger LLM. We'll need to wait for some genuine creative insight into the problem, it can't be brute forced.