LLM Links, 2/28/2025
Scott Belsky on AI agents; Tim B. Lee on Deep Research; Tyler Cowen thinks that AI take-off is slow; Ethan Mollick's latest AI advice
a service that bills itself as “the AI consumer companion” and has tried to productize some of these capabilities - like contesting a parking ticket or getting refunded when your internet goes out. But I suspect many other tools and AI-powered hacks will emerge wherever human friction remains a strategy of sorts for companies and governments. The use cases of AI removing friction to foster equal opportunity and righteousness are seldom discussed but very interesting. The people who suffer the most from intentional friction are often the most disadvantaged (who also have the least time to spare). Let’s hope AI is a source of productive change.
I don’t need an AI agent to book an airline ticket. But an AI agent that will cancel a subscription when the introductory offer runs out? I’d be on it.
Deep Research discovers information in the same iterative manner as human researchers. It will do a search, find a document, and read it. Then it will do another search, find another document, and read that one. As the model reads more documents and learns more about a topic, it is able to refine its search criteria and find documents that didn’t appear in earlier search results. This process—which people sometimes describe as “going down a rabbit hole”—allows Deep Research to gain a much deeper understanding of a subject than was possible with previous AI models.
Lee did his own research by asking nineteen subject-matter experts to submit questions to the deep research models of Open AI and Google. They then judged the answers. The results were mixed, although Lee came away quite bullish on the new approach.
2. Human bottlenecks become more important, the more productive is AI. Let’s say AI increases the rate of good pharma ideas by 10x. Well, until the FDA gets its act together, the relevant constraint is the rate of drug approval, not the rate of drug discovery.
2b. These do not have to be regulatory obstacles, though many are. It may be slow adopters, people in the workplace who hate AI, energy constraints, and much more. It simply is not the case that all workplace inputs are rising in lockstep, quite the contrary.
If we are in a Solow moment with AI (seeing it everywhere but in the productivity statistics), blame humans.
My own view is that when AI is not quite ready for prime time, we disdain it. When it is ready for prime time, it gets embedded in ways that seem incremental. For years, software developers have been doing less routine coding and instead building on existing solutions. The use of the latest AI can be thought of as a continuation of that.
Regardless of what model you pick, you should experiment. Ask the model to code something for you by just asking for it (I asked Claude for a video game with unique mechanics based on the Herman Melville story “Bartleby the Scrivner” - and it did so based on a single prompt), feed it a document and ask it for an infographic summary, or ask it to comment on an image you upload. If this is too playful, follow the advice in my book and just use it for work tasks, taking into account the privacy caveat above. Use it to brainstorm new ideas, ask it how a news article or analyst report might affect your business, or ask it to create a financial dashboard for a new product or startup concept. You will likely find cases that amaze you, and others where the new models are not yet good enough to be helpful.
The challenge is to keep up with an animal that is not standing still.
In terms of AI metrics, I would like to see something different from the current approach. I would like to see AI’s measured against certain desired end states.
For example, one end state would be a “KlingBot” that would mimic my thinking. It could answer the prompt “How would economist Arnold Kling explain the causes of the 2008 financial crisis?” without wrongly injecting other economists’ explanations. It would be able to feed in a substack essay by someone I follow and predict that commentary I would make concerning the essay.
Another end state would be “The Young Lady’s Illustrated Primer.” Neal Stephenson fans will recognize this as the subtitle to The Diamond Age. This would be a mentor/teacher that can adapt to the educational needs of a child as he or she grows up. Progress of an existing AI would be measured in terms of what proportion of a child’s education can be handled by it.
substacks referenced above:
@
@
@
“The Young Lady’s Illustrated Primer” is too rich an allegory to leave hanging.
The Primer is commissioned by a duke-level Equity Lord, Alexander Chung Sik Finkle-McGraw, who has a St. Godric of Finchale type biography, ascending from humble origins into the ruling oligarchy, and along the way develops the shocking non-conformist attitude that “some kinds of behavior were bad, and others good, and it was reasonable to live one’s life accordingly.” The Primer is commissioned for the use of Finkle-McGraw’s grand daughter as an alternative to a public school education on the premise that “in order to raise a generation of children who can reach their full potential, we must find a way to make their lives interesting.”
Early in the book in a bit of inspired foreshadowing, Finkle-McGraw hires Hackworth, an engineer who studied romantic poetry in college but who reads the journals “appropriate to his station,”to build The Primer assigns the build of its power supply to Cotton who immediately encounters problems using “auto assembly” and fixes the problem using a “furiously proscribed” “intuitive approach to the job.”
And these oppositions play themselves out quite interestingly when the Primer falls into the hands of a thete, that is a member of the lower class who was not part of a tribe.
So how is this allegorical?
First, none of the links today make any reference to truth that I noticed. In The Diamond Age, “it is not necessarily a good thing for everyone to read a newspaper in the morning; so the higher one rose in the society, the more similar one’s Times became to one’s peers’.” The research AI’s discussed appear to pretty much function in a similar fashion, reading more papers than humanly possible, prioritizing the most recent, and condensing them down into a received popular wisdom acceptable to the hermetically isolated few. Cowen linked to a piece the other day, Feudalism as a Contested Concept, https://www.broadstreet.blog/p/feudalism-as-a-contested-concept?utm_campaign=post&utm_medium=web that conveys the zeitgeist perfectly. Nothing in the piece itself actually about what is commonly understood to be referenced and its ambiguity by the shorthand term “feudalism” that is not better and more informatively conveyed in the 1911 Encyclopedia Britannica entry for Feudalism, or for that matter in the few short introductory paragraphs of Carl Stephenson’s classic Medieaeval Feudalism, yet the author asserts it is a subject that “non-specialists are not equipped to have an informed opinion” and “it goes without saying that good historical social science should be in dialogue with the most up-to-date historical scholarship.” It is all a realm to which the annointed have exclusive access and apparently more concerned with dialogue between the annointed than the revelation of any truth about that which the humans who lived under what is referenced as “feudalism” experienced. One suspects that researchers might enrich their understandings if with every AI query they ran a separate query restricted to sources dated before 1950. Do AI’s even get trained on such classic sources?
To the extent that AI poses a threat in limiting access to currently unpopular bodies of knowledge, one suspects the world would be better off with a more widespread practice of an intellectual aparigraha, that is is a self-restraint from the culture of exclusivity where one's own professional status as well as material gain or happiness is enhanced by AI gnosticism.
But even in its absence, one suspects that under the new AI overlords and their surveillance state, a new underclass will grow, not embarassed by either culture or routine, and with it will awaken primitive energies in individuals who will find new avenues of commerce and discovery to push aside this establishment, enrich themselves, and in so doing, advance progress across the human race. A new AI enhanced darkweb market with efficacious anonymity tools anyone?
Can AI cause us to feel guilt about our behavior? I would like an AI to monitor my children. In boxing class, their coach provides character education, especially when children in the class misbehave. In fact, this is one of the most valuable aspects of boxing class —my kids gain access to a moral mentor.
He is in the process of creating an app that will allow the parent to teach boxing to kids. But can he mimic his presence through this app? Not yet. That would take a sophisticated AI, monitoring the movements and words of the children.
Surely such an AI system would be a boon to character education, not only in schools, but at home, in churches, on the field, and in daycares. With the press of a button a teacher or parent could command an AI agent to “individualize” on a specific student, beginning a dialogue with him about his misbehavior. This dialogue could be recorded and sent home to his parents for a follow-up conversation.
Not only do we want to mimic Arnold Kling’s mind, we want to mimic Jesus Christ, MLK, and other moral exemplars, not only for our children, but for all of us.
Soon I will be able to hire an AI agent to monitor my behavior. My children will task it to talk to me about my misbehavior. Let’s hope freedom of encryption is amended in the Constitution so that we can keep our private lives private.