LLM Links

Alex Sima on simulating corporate culture; Ethan Mollick on the fastest adopters of LLMs; Andrej Karpathy on RLHF; Mark McNeilly on concerns about people learning bad manners

Alex Sima writes,

What if we structured AI agent interactions like the org charts of big tech companies?

Here are some key takeaways I learned after organizing groups of AI agents as if they were in companies like Apple, Microsoft, Google, Amazon, and more:

1. Companies with multiple “competing” teams (i.e. competing to produce the best final product) like Microsoft and Apple outperformed centralized hierarchies.

2. Systems with single points of failure (for example, one leader making important decisions) like Google, Amazon, and Oracle underperformed.

3. Big-tech organizational structures had a modest but noticeable impact on problem-solving capability.

The use of LLMs as simulation tools has great potential, especially in economics. Recall my recent review of Doyne Farmer’s book.

Ethan Mollick writes,

When I started this blog there were no AI chatbot assistants. Now, all indications that they are likely the fastest-adopted technology in recent history. A survey of 100,000 knowledge workers in Denmark that concluded in January, 2024 found really high adoption rates, as well as high rates of actual use

…research from The Walton Family Foundation finds teachers, parents and students have adopted AI remarkably quickly. Some of this use by students is cheating, of course, a topic I have discussed before, but students, parents and teachers are finding all kinds of other applications as well.

Andrej Karpathy writes,

What would it look like to train AlphaGo with RLHF? Well first, you'd give human labelers two board states from Go, and ask them which one they like better:

Then you'd collect say 100,000 comparisons like this, and you'd train a "Reward Model" (RM) neural network to imitate this human "vibe check" of the board state. You'd train it to agree with the human judgement on average. Once we have a Reward Model vibe check, you run RL with respect to it, learning to play the moves that lead to good vibes. Clearly, this would not have led anywhere too interesting in Go. There are two fundamental, separate reasons for this: 1. The vibes could be misleading - this is not the actual reward (winning the game). This is a crappy proxy objective. But much worse, 2. You'd find that your RL optimization goes off rails as it quickly discovers board states that are adversarial examples to the Reward Model. Remember the RM is a massive neural net with billions of parameters imitating the vibe. There are board states are "out of distribution" to its training data, which are not actually good states, yet by chance they get a very high reward from the RM.

His point is that there is a big difference between reinforcement learning that comes from an objective goal (winning at Go) and passing subjective human tests. Pointer from Tyler Cowen.

Mark McNeilly point to a PopSci article.

In other words, conversational patterns learned while speaking with an AI could then pop-up when that same person holds down a conversation with a human. But speaking with a machine and a human aren’t the same, even if they may sound similar on the surface. OpenAI notes its model is programmed to be deferential to the user, which means it will cede authority and let the user interrupt them and otherwise dictate the conversation. In theory, a user who normalizes conservations with machines could then find themselves interjecting, interrupting, and failing to observe general social cues. Applying the logic of chatbot conversations to humans could make a person awkward, impatient, or just plain rude. 

Somehow, I don’t think that is a big problem. Just have the chatbot say “I was speaking!” in its best girlboss voice.

substacks referenced above:

@

The New News in AI: 8/16/24

@

Change blindness

@

What if AI had corporate culture?