The WSJ hallucinates about LLMs; David Strom on Movable Type's book writer; Henrik Karlsson on humans learning from AI; me on a podcast with Alan Pentz
I think you're read of the shift for Go players is closer to the truth, and I enjoyed reading a more detailed account of your experience. Especially, the phrasing "reconceptualize the game" that I used is too strong, though I get the impression that "abandoning heuristics" applies to some extent, though here we get into murky water and semantics.
I was thinking about this a couple days ago regarding climate change. Journalism is difficult. Journalists are frequently writing on topics they have less than complete knowledge. They are certain to get some things wrong. After all, the supposed experts often don't agree. As a reader, some errors are glaring to the non-expert but many aren't so it is near impossible to know what within the article is true. Nevermind how opinion gets mixed in, made to look like fact.
You would think this could get cleared up quite easily with a sufficiently prominent counter commentary mechanism. Maybe you should start a Substack on this specifically aimed at journalism’s errors in climate change.
Except I'm not an expert. I only know enough to know some of the stuff said isn't true, some stat d as fact is no more than probable, and some isn't even probable. Plenty I don't really know. ... But also wonder if the experts really know.
Regardless, there are plenty of websites trying to dispell CC myths. And just as many opinions on which are biased for, against, or fair in judgement of coverage of the issue.
"What I really want to create is a dedicated GPT that uses those essays and other writing of mine to represent me interactively on the topics addressed there."
You want to do that as an exercise, as a way to generate more new content, or as a way for your readers to eventually be able to answer "what would Arnold Kling say about X"?
the latter. Right now, I have to guess--do I need to keep posting on health care policy, or can I assume people have either read my book or seen my thoughts in other outlets?
A personal corpus analysis tool could be helpful for nudging oneself in the direction of making new points or new insights, to avoid retreading the same ground.
In that case, aren't you worried about the GPT understanding Arnold Kling syntactically but not "semantically"? To oversimplify, say a someone postulates a new idea called Critical Race Theory and GPT has to come up with your opinion on it. Will it just spew a bunch of nonsense based on what it thinks to be your usual response to things Critical, things Race and Theories?
Also, in my experience GPT doesn't "understand" the concept of agency. If it has to take into account the motivations and personal experiences of those postulating this new idea instead of just taking them at their word, will it be able to do so?
"Instead, as my readers know, LLMs take a given word and find patterns of how it is used. What word usually precedes it? What word almost never precedes it? What words earlier in a sentence or paragraph indicate that this word is likely to appear soon? What words earlier in a sentence or paragraph indicate that this word is unlikely to appear soon? These sorts of characteristics are quantified and coded as vectors. What an LLM knows are these vectors."
This is *not* how LLMs work, or at least it is a gross oversimplification, as gross as the WSJ's characterization. You make it sound like LLMs are developed just by tracking the proximity of words and using that to build some sort of probabilistic model. That is not how neural networks work. The only vector that is coded is a set of weights that happen to locally minimize a given loss function.
Why neural networks work so well at building models is a subject of a lot of study, and the true answer is pretty much unknown. Since neural networks were inspired by the brain's architecture, the reason they work so well may well have a lot to do with why brains work the way they do. No one can be sure. What is certain is that LLMs are *not* simply glorified auto-complete. They can perform basic reasoning, including playing chess and even doing mathematics.
It's funny how the WSJ writer seems to be wrong about how LLMs work in exactly the way that would be most damning in a court case where large news papers were suing LLMs makers for copy right infringement on the claims that the LLM will spit out the content of articles verbatim.
What a coincidental error to make, of all the errors possible.
It is very important, and legally important for copyright, that LLMs do NOT remember all written data such that they can print it out verbatim.
Search engines already do that, for every text digitized and available on the net. The probability issue is key, and humans are not so good with probabilities.
Playing games, like writing code, has a very quick feedback of optimal or working according to test case testing. Most “wisdom” and human communication of ideas is not like that. Like attempting to define what a good teacher is, or does. Some want AI to make deepfakes, like of Taylor Swift in porno. (See X, also a different Swift complaint: “She’s a 32 y.o. woman acting like a 16 y.o., beloved by 32 y.o. women wanting to be 16.”)
Brian Chou makes a good case that the George Floyd narrative & response was essentially a deepfake, in getting so mamy to believe, inaccurately, that there were a large number of unarmed Blacks killed by cops.
Part of me wants an aiBot that tells me what Arnold would say on any topic, even after death, but mostly I prefer real people. Except when reading fiction, like Reacher or Strike murders. Tho maybe an aiBot GRR Martin could write a couple more Game Of Thrones books, with a better than TV ending.
I don’t know much about AI in Go or Othello, but I know a bit about Chess. In Chess, there are two types of engines. Traditional engines use alpha beta pruning (or some other minimax algorithm) with a pre-programmed heuristic evaluation function that give a mathematical evaluation of the position. Greater than 0.0 favors white, less than 0.0 favors black. In the 90s and 00s, engines could generally play at a professional or better level, but there were still places where humans had better understanding, particularly in closed positions where both sides could shuffle pieces for a long time so that the plans would extend beyond the engines calculation, known as the horizon effect. By the 10s, traditional engines generally played better than humans in all situations. Go engines at this time were strong, but not as strong as the best humans because the size of the board and the number of possible moves in a single position made deep calculation with alpha beta pruning impractical.
In 2017, Google released some games between Alpha Zero, a successor to AlphaGo, and the top traditional engine, Stockfish. Alpha Zero doesn’t use an alpha beta pruning search tree; it uses what is called a Monte Carlo Tree Search combined with a neural network, similar in structure to what LLMs use. Basically, Alpha Zero combined a random move generator with a neural network to play out billions of games against itself to teach itself how to play and to generate a set of probabilities in a given position for each candidate move. Instead of a quantitative evaluation, Alpha Zero gives a probability of one side or the other winning, based on its experience playing itself.
Chess engines were already much stronger than humans, but this was an immediate and massive leap in strength over existing engines and was immediately followed by various copies, the open source Leela Zero being the most successful pure neural network engine. Now, all the top engines, like Stockfish, combine aspects of both types. Interestingly, the new neural network engines were perceived to play ‘more humanly’ in the sense that their play required less depth of calculation and the moves played seemed more amenable to verbal explanations compared to the traditional engines.
There wasn’t a huge revolution in human chess, but there was clearly a large effect. The new engines were far less materialistic, much more focused on piece activity, and willing to push flank pawns to gain space, especially in front of the opponents king. Pushing the h pawn to the h6 square was something strong players like Kasparov had done some in the past, but it’s become much more common since and is something of a trademark of the new engines.
I think the number of surprising engine suggestions is much higher in Chess than in Othello. It’s very common for engines to recommend moves that look unintuitive due to a number of cognitive biases in humans. For example, humans are less likely to find strong backwards moves when attacking. However, this isn’t a revolution in strategic understanding the game as much as the difficulties in calculating a complex position with lots of variables. Chess players distinguish between tactics and strategy and the bulk of surprising computer recommendations are tactical rather than strategic.
> For many decades, it seemed professional Go players had reached a hard limit on how well it is possible to play. They were not getting better. Decision quality was largely plateaued from 1950 to the mid-2010s
I don't know what "decision quality" is, but there was at least one Go revolution in this interval: in the 90s when the younger generation of Koreans starting with Lee Chang Ho came into their full powers. They developed a lot of new stuff, both in the openings and in general theory, and the style of play of strongest players changed accordingly because it was simply better. They also knocked the socks off Japanese professionals, whereas before the 90s it was the Koreans who went to learn Go in Japan.
“I see AI as a software tool” was one of the most valuable statements in the podcast. This seems obvious, but it’s surprising how engineers and technologists get so discombobulated in killer AI discourse.
Thanks for the podcast on AI. I liked your descriptions of the AI mentor and teacher concept that would follow a child around. I would call this Father AI. Its main purpose would be to motivate the child to learn and grow. I see this as a great opportunity to introduce the “impartial spectator” concept into AI technology in which life long learning and virtuous leadership become ultimate goals. The best description of what this looks like is written down here in The Thales Way by Bob Luddy.
I think you're read of the shift for Go players is closer to the truth, and I enjoyed reading a more detailed account of your experience. Especially, the phrasing "reconceptualize the game" that I used is too strong, though I get the impression that "abandoning heuristics" applies to some extent, though here we get into murky water and semantics.
Errors in journalism
I was thinking about this a couple days ago regarding climate change. Journalism is difficult. Journalists are frequently writing on topics they have less than complete knowledge. They are certain to get some things wrong. After all, the supposed experts often don't agree. As a reader, some errors are glaring to the non-expert but many aren't so it is near impossible to know what within the article is true. Nevermind how opinion gets mixed in, made to look like fact.
You would think this could get cleared up quite easily with a sufficiently prominent counter commentary mechanism. Maybe you should start a Substack on this specifically aimed at journalism’s errors in climate change.
Except I'm not an expert. I only know enough to know some of the stuff said isn't true, some stat d as fact is no more than probable, and some isn't even probable. Plenty I don't really know. ... But also wonder if the experts really know.
Regardless, there are plenty of websites trying to dispell CC myths. And just as many opinions on which are biased for, against, or fair in judgement of coverage of the issue.
It would be great to get these climate change websites onto Substack if they aren’t already.
Watt's Up With That, Judith Curry & some others whose names escape me at the moment have long established websites. WUWT gets heavy traffic.
Good memory. Here’s my 15 minute effort. First cut. Will look on Substack later.
https://www.drroyspencer.com
https://www.nsstc.uah.edu/users/john.christy/
https://energytalkingpoints.com
https://lomborg.com
https://judithcurry.com
https://www.amazon.com/Whole-Story-Climate-Science-Reveals/dp/1633886026
https://wattsupwiththat.com
https://emanuel.mit.edu
https://www.econtalk.org/matt-ridley-on-climate-change/
https://www.econtalk.org/robert-pindyck-on-averting-and-adapting-to-climate-change/
https://www.amazon.com/Role-Sun-Climate-Change/dp/019509414X
"What I really want to create is a dedicated GPT that uses those essays and other writing of mine to represent me interactively on the topics addressed there."
You want to do that as an exercise, as a way to generate more new content, or as a way for your readers to eventually be able to answer "what would Arnold Kling say about X"?
the latter. Right now, I have to guess--do I need to keep posting on health care policy, or can I assume people have either read my book or seen my thoughts in other outlets?
A personal corpus analysis tool could be helpful for nudging oneself in the direction of making new points or new insights, to avoid retreading the same ground.
In that case, aren't you worried about the GPT understanding Arnold Kling syntactically but not "semantically"? To oversimplify, say a someone postulates a new idea called Critical Race Theory and GPT has to come up with your opinion on it. Will it just spew a bunch of nonsense based on what it thinks to be your usual response to things Critical, things Race and Theories?
Also, in my experience GPT doesn't "understand" the concept of agency. If it has to take into account the motivations and personal experiences of those postulating this new idea instead of just taking them at their word, will it be able to do so?
Don’t assume we’ve read your books. We just discovered you.
"Instead, as my readers know, LLMs take a given word and find patterns of how it is used. What word usually precedes it? What word almost never precedes it? What words earlier in a sentence or paragraph indicate that this word is likely to appear soon? What words earlier in a sentence or paragraph indicate that this word is unlikely to appear soon? These sorts of characteristics are quantified and coded as vectors. What an LLM knows are these vectors."
This is *not* how LLMs work, or at least it is a gross oversimplification, as gross as the WSJ's characterization. You make it sound like LLMs are developed just by tracking the proximity of words and using that to build some sort of probabilistic model. That is not how neural networks work. The only vector that is coded is a set of weights that happen to locally minimize a given loss function.
Why neural networks work so well at building models is a subject of a lot of study, and the true answer is pretty much unknown. Since neural networks were inspired by the brain's architecture, the reason they work so well may well have a lot to do with why brains work the way they do. No one can be sure. What is certain is that LLMs are *not* simply glorified auto-complete. They can perform basic reasoning, including playing chess and even doing mathematics.
It's funny how the WSJ writer seems to be wrong about how LLMs work in exactly the way that would be most damning in a court case where large news papers were suing LLMs makers for copy right infringement on the claims that the LLM will spit out the content of articles verbatim.
What a coincidental error to make, of all the errors possible.
It is very important, and legally important for copyright, that LLMs do NOT remember all written data such that they can print it out verbatim.
Search engines already do that, for every text digitized and available on the net. The probability issue is key, and humans are not so good with probabilities.
Playing games, like writing code, has a very quick feedback of optimal or working according to test case testing. Most “wisdom” and human communication of ideas is not like that. Like attempting to define what a good teacher is, or does. Some want AI to make deepfakes, like of Taylor Swift in porno. (See X, also a different Swift complaint: “She’s a 32 y.o. woman acting like a 16 y.o., beloved by 32 y.o. women wanting to be 16.”)
Brian Chou makes a good case that the George Floyd narrative & response was essentially a deepfake, in getting so mamy to believe, inaccurately, that there were a large number of unarmed Blacks killed by cops.
Part of me wants an aiBot that tells me what Arnold would say on any topic, even after death, but mostly I prefer real people. Except when reading fiction, like Reacher or Strike murders. Tho maybe an aiBot GRR Martin could write a couple more Game Of Thrones books, with a better than TV ending.
I don’t know much about AI in Go or Othello, but I know a bit about Chess. In Chess, there are two types of engines. Traditional engines use alpha beta pruning (or some other minimax algorithm) with a pre-programmed heuristic evaluation function that give a mathematical evaluation of the position. Greater than 0.0 favors white, less than 0.0 favors black. In the 90s and 00s, engines could generally play at a professional or better level, but there were still places where humans had better understanding, particularly in closed positions where both sides could shuffle pieces for a long time so that the plans would extend beyond the engines calculation, known as the horizon effect. By the 10s, traditional engines generally played better than humans in all situations. Go engines at this time were strong, but not as strong as the best humans because the size of the board and the number of possible moves in a single position made deep calculation with alpha beta pruning impractical.
In 2017, Google released some games between Alpha Zero, a successor to AlphaGo, and the top traditional engine, Stockfish. Alpha Zero doesn’t use an alpha beta pruning search tree; it uses what is called a Monte Carlo Tree Search combined with a neural network, similar in structure to what LLMs use. Basically, Alpha Zero combined a random move generator with a neural network to play out billions of games against itself to teach itself how to play and to generate a set of probabilities in a given position for each candidate move. Instead of a quantitative evaluation, Alpha Zero gives a probability of one side or the other winning, based on its experience playing itself.
Chess engines were already much stronger than humans, but this was an immediate and massive leap in strength over existing engines and was immediately followed by various copies, the open source Leela Zero being the most successful pure neural network engine. Now, all the top engines, like Stockfish, combine aspects of both types. Interestingly, the new neural network engines were perceived to play ‘more humanly’ in the sense that their play required less depth of calculation and the moves played seemed more amenable to verbal explanations compared to the traditional engines.
There wasn’t a huge revolution in human chess, but there was clearly a large effect. The new engines were far less materialistic, much more focused on piece activity, and willing to push flank pawns to gain space, especially in front of the opponents king. Pushing the h pawn to the h6 square was something strong players like Kasparov had done some in the past, but it’s become much more common since and is something of a trademark of the new engines.
I think the number of surprising engine suggestions is much higher in Chess than in Othello. It’s very common for engines to recommend moves that look unintuitive due to a number of cognitive biases in humans. For example, humans are less likely to find strong backwards moves when attacking. However, this isn’t a revolution in strategic understanding the game as much as the difficulties in calculating a complex position with lots of variables. Chess players distinguish between tactics and strategy and the bulk of surprising computer recommendations are tactical rather than strategic.
> For many decades, it seemed professional Go players had reached a hard limit on how well it is possible to play. They were not getting better. Decision quality was largely plateaued from 1950 to the mid-2010s
I don't know what "decision quality" is, but there was at least one Go revolution in this interval: in the 90s when the younger generation of Koreans starting with Lee Chang Ho came into their full powers. They developed a lot of new stuff, both in the openings and in general theory, and the style of play of strongest players changed accordingly because it was simply better. They also knocked the socks off Japanese professionals, whereas before the 90s it was the Koreans who went to learn Go in Japan.
My sermon for today. Too long for the comments section.
https://open.substack.com/pub/scottgibb/p/discourse-platforms?r=nb3bl&utm_campaign=post&utm_medium=web
You could use PoSE and Llama2 to create an AI to query about your essays.
https://arxiv.org/pdf/2309.10400v1.pdf
“I see AI as a software tool” was one of the most valuable statements in the podcast. This seems obvious, but it’s surprising how engineers and technologists get so discombobulated in killer AI discourse.
As I have written before, I will get excited (and worried, too) about A.I. when it solves a math problem that hasn't been solved by humans.
Thanks for the podcast on AI. I liked your descriptions of the AI mentor and teacher concept that would follow a child around. I would call this Father AI. Its main purpose would be to motivate the child to learn and grow. I see this as a great opportunity to introduce the “impartial spectator” concept into AI technology in which life long learning and virtuous leadership become ultimate goals. The best description of what this looks like is written down here in The Thales Way by Bob Luddy.
https://www.thalesacademy.org/assets/docs/the-thales-way-bob-luddy.pdf