Made-up Training Data; Ethan Mollick on how to learn to use LLMs; Mark Wilson on Dot, a companion chatbot; The Zvi on people who say ho-hum about AI; Create your own GPT!
> just using AI will teach you how to use AI. You can become a world expert in the application of AI to your domain by just using AI a lot until you figure out what it is good and bad at.
Sounds like Ethan Mollick is more talented at this than I am. The more I use AI the more I become like a grumpy old man who thinks this new fangled stuff never works. But I still cling to the hope that I'm just holding it wrong.
I think the sychopancy issue is going to conflict a lot with the plan to use AI to grade the reasoning of essays, or for automating FIT scores, etc. The use case relies on the AI understanding (for lack of a better word) the content of the essay, but the AIs understanding of the content and how it responds are going to be shaded by the training and the user. For instance, I would be worried that it would grade essays based on me on my worst morning I know I grade differently depending on how I am feeling, and have learned to recognize when I am not in a good place for it and come back to grading later. Likewise, we have all had professors who have given good grades for parroting back their preferred position while giving bad grades even for perfectly well argued points they disagree with. If an AI exhibits the same behavior that is worrisome, because people kind of understand the "professor just wants you to agree with him" situation, but will inevitably say "Hey, it was a fair and honest machine, it can't be prejudiced." In other words, it will merely be another source of confirmation bias.
> [not] trying to understand what they are looking at
Sometimes people are ignorant about something because they were indifferent to it. Not simply: prepared to like it ... but underwhelmed by the results, nor - prepared to really be keen on it, but let down by their own failure of imagination or aptitude.
A failure of imagination applies to 99% of us 99% of the time, so that can hardly be a bar for judging the importance of anything.
Indifference is a phenomenon that might best be sympathized with in the obverse.
Some of us who do not care very much about "online" generally feel a great interest in the real world of things - and the steady diminishment of interest in the physical world over one's adult life is felt as an almost unimaginably frustrating loss.
It should be enough that one side is clearly winning. Humility should allow that it's okay that there are two camps. It would, in this view which is admittedly not your own, be very disturbing if "everyone" really liked this AI ... stuff. If there was no subset of "smart" people who cared about it not one whit.
"A behavior known as sycophancy" :) I like this framing for social media and the broader info environment- we're all in a large RLHF training run, and we're each both trainee and reinforcer.
Sycophancy attracts us toward coordination points, tending to 2 clusters (red/blue).
Bryan Caplan recently framed the Great Man Theory of history in game theory/coordination point terms. So maybe if we're all orbiting each other, drawn in by the sycophancy force, those who pull the orbital center closer to themselves are "great" - though that's a temporary equilibrium.
I find #3 of Kruel's list particularly terrifying somehow:
> “Can we make LLMs unlearn a subset of their training data? ...we took Llama2-7b and in 30 minutes of fine-tuning, made it forget the Harry Potter universe while keeping its performance on common benchmarks intact...” https://www.microsoft.com/en-us/research/project/physics-of-agi/articles/whos-harry-potter-making-llms-forget-2/
> just using AI will teach you how to use AI. You can become a world expert in the application of AI to your domain by just using AI a lot until you figure out what it is good and bad at.
Sounds like Ethan Mollick is more talented at this than I am. The more I use AI the more I become like a grumpy old man who thinks this new fangled stuff never works. But I still cling to the hope that I'm just holding it wrong.
I think the sychopancy issue is going to conflict a lot with the plan to use AI to grade the reasoning of essays, or for automating FIT scores, etc. The use case relies on the AI understanding (for lack of a better word) the content of the essay, but the AIs understanding of the content and how it responds are going to be shaded by the training and the user. For instance, I would be worried that it would grade essays based on me on my worst morning I know I grade differently depending on how I am feeling, and have learned to recognize when I am not in a good place for it and come back to grading later. Likewise, we have all had professors who have given good grades for parroting back their preferred position while giving bad grades even for perfectly well argued points they disagree with. If an AI exhibits the same behavior that is worrisome, because people kind of understand the "professor just wants you to agree with him" situation, but will inevitably say "Hey, it was a fair and honest machine, it can't be prejudiced." In other words, it will merely be another source of confirmation bias.
> It takes imagination to come up with a use case
> [not] trying to understand what they are looking at
Sometimes people are ignorant about something because they were indifferent to it. Not simply: prepared to like it ... but underwhelmed by the results, nor - prepared to really be keen on it, but let down by their own failure of imagination or aptitude.
A failure of imagination applies to 99% of us 99% of the time, so that can hardly be a bar for judging the importance of anything.
Indifference is a phenomenon that might best be sympathized with in the obverse.
Some of us who do not care very much about "online" generally feel a great interest in the real world of things - and the steady diminishment of interest in the physical world over one's adult life is felt as an almost unimaginably frustrating loss.
It should be enough that one side is clearly winning. Humility should allow that it's okay that there are two camps. It would, in this view which is admittedly not your own, be very disturbing if "everyone" really liked this AI ... stuff. If there was no subset of "smart" people who cared about it not one whit.
"A behavior known as sycophancy" :) I like this framing for social media and the broader info environment- we're all in a large RLHF training run, and we're each both trainee and reinforcer.
Sycophancy attracts us toward coordination points, tending to 2 clusters (red/blue).
Bryan Caplan recently framed the Great Man Theory of history in game theory/coordination point terms. So maybe if we're all orbiting each other, drawn in by the sycophancy force, those who pull the orbital center closer to themselves are "great" - though that's a temporary equilibrium.