The example makes clear why you tightened the tutor’s constraints. The earlier version was doing good conceptual work, but the verbosity diluted the signal. Targeted insights plus disciplined Socratic prompts seem like the right balance for preparing students before class.
A few months ago, I messed around with creating a Custom GPT based on my books and writings. I was hoping readers could ask questions, and then it would answer as if it were me based on my writings.
I eventually gave up because no matter how many times I told it to use specific terms and phrases, it would always add lib in ways that twisted the meaning. And that was even when all my books and Substack articles were stored in its memory.
For those of us not on the left, obtaining alignment from an AI can be a challenge. I am not as put off by it as you were. Some possible reasons:
(1) using a recent version of Claude, and a paid version at that (2) obtaining agreement beforehand about the main points I want the student to get. For each chapter, you might have a conversation about what the key points are. Have the AI state the main points, and make sure that it gets them the way that you want. I'm going to do that for all of the readings in my course. (3) stating general principles that help with alignment. You might say that as a general principle you want phrases that come from your writings.
It wasn’t really ideological bias. It just ad libbed.
For example, I would give it a specific definition of a term and tell it to use the term in only that way. Then layer I would ask it what that term meant, and it would give different reply that was largely correct but not precise enough.
I actually had a conversation with ChatGPT as to why it would not follow specific instructions.
It was really more about memory storage. It does not store exact text in memory, even when you tell it to. It reads the text, summarizes it, and then when necessary, it retrieves the summary and paraphrases the summary.
It is like the telephone game. I think it is a form of energy economizing.
That might be fine for your purposes, but I lost confidence that it could really answer questions as if it were me.
I loved your post. The Socratic exchange reminded me of my law school days during our first year in my small group class. Our professor would actually challenge each student to use their own words (not parrot back the words we were reading in the cases). It could be frustrating but it helped me to better understand the essence of the legal principles rather than just memorizing the buzzwords.
I read some of the comments about the AI dialogue. I actually think the repetition helps to reinforce the point of the lesson. When you actually understand a concept, you fully digest it and can distill an abstract idea without repeating it or using a synonymous concept (e.g., freedom/liberty).
PS—Freedom can be defined by Newton’s first law: a body at rest remains at rest, or if in motion, remains in motion unless acted on by an external force.
So the essence of freedom is no external force. An individual cannot be forced or compelled to act by others. In other words, free individuals have no rulers—whether they be kings or people’s democracies.
Like Duane, I cannot get anything at the link. A few things--- Claude itself tells me this is likely not the workflow you want to use for this. It suggests using projects rather than the skill function. I don't know, it just kicked that out when I was trying to figure out what to do on my end. Likely worth a double check.
That output in the prior example seems like 100 percent boilerplate LLM. Not seeing any glimmers of Kling in the bot.
I love this idea and will give it a go. I liked the idea of your Oberlin simulation, but found the classroom dynamics more distracting than immersive. I think a one-on-one socratic vibe reading sounds amazing. If you can port your other projects topics over into this format I can see the results being amazing.
Look forward to trying and happy to share the "beta test" results
As this is clearly development, it’s better to think of it as alpha testing.
The beta testing starts when AK thinks it’s good enough to be released, so changes are frozen, added to the expanded list of changes desired.
Alpha testers have much more input into possible design changes & style choices, like whether the tutor’s answer is too long or too incomplete. Very important for development to get early feedback.
On one hand, throwing the reading of source materials out the window and instead having students acquire a cliff notes understanding of assigned materials would seem vulnerable to to the negative outcomes associated with certain types of cognitive offloading that we see so much about in the press lately. Google scholar produces 1900 results since 2026 (not including citations) for a “negative outcomes cognitive offloading” search. Of course not all those results were reporting negative outcomes, but a quick scan of a handful suggested potential concerns about the impact on memory of AI use and also declines in self confidence in AI using students. Given Dunning-Kruger, a decline in self-confidence may or may not be a negative thing.
On the other hand, Google scholar produced 2,270 results since 2026 (no citations) for “Socratic questioning AI.” A quick scan of a handful of those results suggested mostly positive outcomes. One wonders, though, whether we ought distinguish critical thinking producing Socratic questioning from something more along behaviorist operant conditioning lines. It wasn’t clear to me that all of the results that I looked at would qualify as Socratic in intent or methodology.
So, some questions it might be interesting to investigate:
Do students who read source materials have better long term recall than those that use the AI cliff notes approach?
Does the AI questioning model produce transferable learning skills?
Can the AI questioning model promote a student’s ability to defend a controversial position?
Are there sex-based differences in AI questioning model learning outcomes, particularly longer term, in self-confidence in mastery of material and persuasive argumentation?
“Do students who read source materials have better long term recall than those that use the AI cliff notes approach?“
Short answer: not for the average person.
The problem with complex texts for average IQ people like myself is that we get lost in the trees trying to understand the nuance of every idea and miss the larger context of the forest.
For example, Das Kapital is 1300+ pages. I’m far more likely to remember a great summary of it over having ventured deep into the weeds. In that sense, there is something special about AI for many people like myself - the complex can be distilled down to something more accessible and then understood in further detail with follow up questions.
Oh I have no doubt that you are of above average intelligence and likely higher. But you are correct. I think the gist of the research on the matter is that assigning large chunks of a textbook as a reading assignment does nothing for retention and can leave students unclear on what they are supposed to be learning. And it might well be overkill to assign one of Lakoff’s books just to get at the strict father model. But Hoffer is only 176 pages and you probably don’t need to read the whole thing. And isn’t there something to be said for experiencing the author’s voice?
I’ve never attempted all 3000 pages of Das Kapital (kindle version) but I’m glad I’ve poked around the internet enough to be exposed to some of his prose, which whether agree, not, or don’t know, is memorable in itself and some direct exposure to Marx would seem essential to getting a sense of his spirit. Stuff like “"Accumulate, accumulate! That is Moses and the prophets!" or "the country that is more developed industrially only shows, to the less developed, the image of its own future" I’m guessing give more of a taste of what Marx was about than any second hand recapitulation.
Reading did not come easy to me as a kid and I wound up dropping out of college for a while, so I may not even be of average intelligence. Yet, reading was rewarding and I spent years rereading paragraphs and pages multiple times as my concentration drifted off and I lost the thread. But I stuck with it. And even then when I did make it through classics like Moby Dick and War and Peace, it was not until I had plowed through them second and third times that I really began to appreciate them and attain a sense of reward for engaging with them. And I read to my children and encouraged them to engage with lengthy works and we enjoyed talking about them. To this day, I still talk with them once a week and one of the topics we cover is what they are reading. Whether I should be proud or not, I don’t know, but as of Sunday one child was somewhere in The Magic Mountain and the other in Augustine’s City of God. Of my many failings as a parent, I don’t think encouraging reading was one of them.
So perhaps I take antipathy to reading and books in general as something of a personal affront. Of all the things I have to regret in this life, I can’t really bring myself to regard reading books as one of them. It would be sad to me if someone who might have found reading a great book rewarding didn’t do so because they either never learned to engage with long and difficult writing or because they thought they could save time and just have the AI walk them through it. Oh well, once upon a time I got my hands dirty repairing my own car too so maybe an AI optimized future won’t be as sterile and facile as I tend to fear.
If I recall correctly, you once read and commented on “Power without Knowledge: A Critique of Technocracy” by the now late Jeffrey Friedman. RIP.
He was a legend in my eyes, but this book was intentionally inaccessible for most people. AI solves this, although it is, of course, a poor substitute for the real thing for those with the ability to make the slog. I’m unfortunately not one of those people.
It's great that AI opens up otherwise inaccessible works or allows people to get the most bang for their limited amount of study time. I imagine though that as reading books becomes more of a non-mainstream cultish subversive activity and Fahrenheit 451 is brought on by the legions now of college graduates who have never completely read a novel, the frisson of the forbidden will lead to a rediscovery of readings' pleasures and people's trade off decisions might respond. I seem to recall that 50 percent of college graduates never pick up a book again after they leave school. Maybe subsidizing college so much isn't the best of all possible trade offs?
I hadn't thought of Power without Knowledge for quite a while so I pulled it up again on my kindle and was glad of the annotations I had made and also his fulsome footnotes which I had used to chase down rabbit holes. The technology is wonderful and a book whose footnotes alone can engross you for hours and lead to much note-taking is my kind of book.
Efficient? No. Time well spent? YMMV. Good for fighting off dementia? Hopefully. And much much better to use AI to tap into it than to not tap into it at all.
Interesting. I have similar conversations with Claude about medicine and biology. The conversation does seem quite verbose.
Rather than seeming to require strict conformance to Lakoff's views, I would encourage the student to analyze how their views of human nature differ from those of Lakoff. You could also toss Hoffer and Sowell into the mix if desired.
Accounting to Claude: There is no "view from nowhere." Objectivism — the idea that reason can operate independently of the reasoner's embodied situation — is, for Lakoff, simply false. All reason is perspectival, but that doesn't collapse into relativism because embodied experience provides genuine, shared constraints.
His broader project is a direct challenge to the Enlightenment picture of humans as rational, disembodied agents — and by extension, to much of analytic philosophy, classical economics, and cognitive science built on that foundation.
Me again
So if we're just products of our experience, where's does morality come into play? How did Lakoff come up with his view, was it preordained by his experience?
This sounds to me very much like the bull sessions my friends and I had on Friday nights in college.
I wrote a long comment, briefly went to Claude, tried to come back, and my current had disappeared in spite of my leaving Substack open. Substack needs to work on that.
Arnold, I added the Skill to my Claude Cowork and it works like a charm. Extremely interesting. I'll enjoy chatting with it.
The example makes clear why you tightened the tutor’s constraints. The earlier version was doing good conceptual work, but the verbosity diluted the signal. Targeted insights plus disciplined Socratic prompts seem like the right balance for preparing students before class.
Good luck with this project.
A few months ago, I messed around with creating a Custom GPT based on my books and writings. I was hoping readers could ask questions, and then it would answer as if it were me based on my writings.
I eventually gave up because no matter how many times I told it to use specific terms and phrases, it would always add lib in ways that twisted the meaning. And that was even when all my books and Substack articles were stored in its memory.
For those of us not on the left, obtaining alignment from an AI can be a challenge. I am not as put off by it as you were. Some possible reasons:
(1) using a recent version of Claude, and a paid version at that (2) obtaining agreement beforehand about the main points I want the student to get. For each chapter, you might have a conversation about what the key points are. Have the AI state the main points, and make sure that it gets them the way that you want. I'm going to do that for all of the readings in my course. (3) stating general principles that help with alignment. You might say that as a general principle you want phrases that come from your writings.
It wasn’t really ideological bias. It just ad libbed.
For example, I would give it a specific definition of a term and tell it to use the term in only that way. Then layer I would ask it what that term meant, and it would give different reply that was largely correct but not precise enough.
I actually had a conversation with ChatGPT as to why it would not follow specific instructions.
It was really more about memory storage. It does not store exact text in memory, even when you tell it to. It reads the text, summarizes it, and then when necessary, it retrieves the summary and paraphrases the summary.
It is like the telephone game. I think it is a form of energy economizing.
That might be fine for your purposes, but I lost confidence that it could really answer questions as if it were me.
In that case, I have an advantage working with a paid version of Claude and using its skill function. It doesn't forget what I tell it.
If the skill function can allow Claude to not forget some gigabytes of text, so as to accurately answer as Kling clone, that would be great.
Does it? How do we know, testing is ornerous.
I loved your post. The Socratic exchange reminded me of my law school days during our first year in my small group class. Our professor would actually challenge each student to use their own words (not parrot back the words we were reading in the cases). It could be frustrating but it helped me to better understand the essence of the legal principles rather than just memorizing the buzzwords.
I read some of the comments about the AI dialogue. I actually think the repetition helps to reinforce the point of the lesson. When you actually understand a concept, you fully digest it and can distill an abstract idea without repeating it or using a synonymous concept (e.g., freedom/liberty).
PS—Freedom can be defined by Newton’s first law: a body at rest remains at rest, or if in motion, remains in motion unless acted on by an external force.
So the essence of freedom is no external force. An individual cannot be forced or compelled to act by others. In other words, free individuals have no rulers—whether they be kings or people’s democracies.
Frankly, if I had to go through all that before each session I’d resign! There is something deadening about these long Chat encounters.
That is why I have the new version. The encounter should be shorter.
Look forward to seeing it, interesting project.
Like Duane, I cannot get anything at the link. A few things--- Claude itself tells me this is likely not the workflow you want to use for this. It suggests using projects rather than the skill function. I don't know, it just kicked that out when I was trying to figure out what to do on my end. Likely worth a double check.
That output in the prior example seems like 100 percent boilerplate LLM. Not seeing any glimmers of Kling in the bot.
I love this idea and will give it a go. I liked the idea of your Oberlin simulation, but found the classroom dynamics more distracting than immersive. I think a one-on-one socratic vibe reading sounds amazing. If you can port your other projects topics over into this format I can see the results being amazing.
Look forward to trying and happy to share the "beta test" results
As this is clearly development, it’s better to think of it as alpha testing.
The beta testing starts when AK thinks it’s good enough to be released, so changes are frozen, added to the expanded list of changes desired.
Alpha testers have much more input into possible design changes & style choices, like whether the tutor’s answer is too long or too incomplete. Very important for development to get early feedback.
On one hand, throwing the reading of source materials out the window and instead having students acquire a cliff notes understanding of assigned materials would seem vulnerable to to the negative outcomes associated with certain types of cognitive offloading that we see so much about in the press lately. Google scholar produces 1900 results since 2026 (not including citations) for a “negative outcomes cognitive offloading” search. Of course not all those results were reporting negative outcomes, but a quick scan of a handful suggested potential concerns about the impact on memory of AI use and also declines in self confidence in AI using students. Given Dunning-Kruger, a decline in self-confidence may or may not be a negative thing.
On the other hand, Google scholar produced 2,270 results since 2026 (no citations) for “Socratic questioning AI.” A quick scan of a handful of those results suggested mostly positive outcomes. One wonders, though, whether we ought distinguish critical thinking producing Socratic questioning from something more along behaviorist operant conditioning lines. It wasn’t clear to me that all of the results that I looked at would qualify as Socratic in intent or methodology.
So, some questions it might be interesting to investigate:
Do students who read source materials have better long term recall than those that use the AI cliff notes approach?
Does the AI questioning model produce transferable learning skills?
Can the AI questioning model promote a student’s ability to defend a controversial position?
Are there sex-based differences in AI questioning model learning outcomes, particularly longer term, in self-confidence in mastery of material and persuasive argumentation?
“Do students who read source materials have better long term recall than those that use the AI cliff notes approach?“
Short answer: not for the average person.
The problem with complex texts for average IQ people like myself is that we get lost in the trees trying to understand the nuance of every idea and miss the larger context of the forest.
For example, Das Kapital is 1300+ pages. I’m far more likely to remember a great summary of it over having ventured deep into the weeds. In that sense, there is something special about AI for many people like myself - the complex can be distilled down to something more accessible and then understood in further detail with follow up questions.
PS welcome back edgar!
Oh I have no doubt that you are of above average intelligence and likely higher. But you are correct. I think the gist of the research on the matter is that assigning large chunks of a textbook as a reading assignment does nothing for retention and can leave students unclear on what they are supposed to be learning. And it might well be overkill to assign one of Lakoff’s books just to get at the strict father model. But Hoffer is only 176 pages and you probably don’t need to read the whole thing. And isn’t there something to be said for experiencing the author’s voice?
I’ve never attempted all 3000 pages of Das Kapital (kindle version) but I’m glad I’ve poked around the internet enough to be exposed to some of his prose, which whether agree, not, or don’t know, is memorable in itself and some direct exposure to Marx would seem essential to getting a sense of his spirit. Stuff like “"Accumulate, accumulate! That is Moses and the prophets!" or "the country that is more developed industrially only shows, to the less developed, the image of its own future" I’m guessing give more of a taste of what Marx was about than any second hand recapitulation.
Reading did not come easy to me as a kid and I wound up dropping out of college for a while, so I may not even be of average intelligence. Yet, reading was rewarding and I spent years rereading paragraphs and pages multiple times as my concentration drifted off and I lost the thread. But I stuck with it. And even then when I did make it through classics like Moby Dick and War and Peace, it was not until I had plowed through them second and third times that I really began to appreciate them and attain a sense of reward for engaging with them. And I read to my children and encouraged them to engage with lengthy works and we enjoyed talking about them. To this day, I still talk with them once a week and one of the topics we cover is what they are reading. Whether I should be proud or not, I don’t know, but as of Sunday one child was somewhere in The Magic Mountain and the other in Augustine’s City of God. Of my many failings as a parent, I don’t think encouraging reading was one of them.
So perhaps I take antipathy to reading and books in general as something of a personal affront. Of all the things I have to regret in this life, I can’t really bring myself to regard reading books as one of them. It would be sad to me if someone who might have found reading a great book rewarding didn’t do so because they either never learned to engage with long and difficult writing or because they thought they could save time and just have the AI walk them through it. Oh well, once upon a time I got my hands dirty repairing my own car too so maybe an AI optimized future won’t be as sterile and facile as I tend to fear.
If I recall correctly, you once read and commented on “Power without Knowledge: A Critique of Technocracy” by the now late Jeffrey Friedman. RIP.
He was a legend in my eyes, but this book was intentionally inaccessible for most people. AI solves this, although it is, of course, a poor substitute for the real thing for those with the ability to make the slog. I’m unfortunately not one of those people.
https://www.amazon.com/Power-without-Knowledge-Critique-Technocracy/dp/0190877170
It's great that AI opens up otherwise inaccessible works or allows people to get the most bang for their limited amount of study time. I imagine though that as reading books becomes more of a non-mainstream cultish subversive activity and Fahrenheit 451 is brought on by the legions now of college graduates who have never completely read a novel, the frisson of the forbidden will lead to a rediscovery of readings' pleasures and people's trade off decisions might respond. I seem to recall that 50 percent of college graduates never pick up a book again after they leave school. Maybe subsidizing college so much isn't the best of all possible trade offs?
I hadn't thought of Power without Knowledge for quite a while so I pulled it up again on my kindle and was glad of the annotations I had made and also his fulsome footnotes which I had used to chase down rabbit holes. The technology is wonderful and a book whose footnotes alone can engross you for hours and lead to much note-taking is my kind of book.
Efficient? No. Time well spent? YMMV. Good for fighting off dementia? Hopefully. And much much better to use AI to tap into it than to not tap into it at all.
Interesting. I have similar conversations with Claude about medicine and biology. The conversation does seem quite verbose.
Rather than seeming to require strict conformance to Lakoff's views, I would encourage the student to analyze how their views of human nature differ from those of Lakoff. You could also toss Hoffer and Sowell into the mix if desired.
Accounting to Claude: There is no "view from nowhere." Objectivism — the idea that reason can operate independently of the reasoner's embodied situation — is, for Lakoff, simply false. All reason is perspectival, but that doesn't collapse into relativism because embodied experience provides genuine, shared constraints.
His broader project is a direct challenge to the Enlightenment picture of humans as rational, disembodied agents — and by extension, to much of analytic philosophy, classical economics, and cognitive science built on that foundation.
Me again
So if we're just products of our experience, where's does morality come into play? How did Lakoff come up with his view, was it preordained by his experience?
This sounds to me very much like the bull sessions my friends and I had on Friday nights in college.
I wrote a long comment, briefly went to Claude, tried to come back, and my current had disappeared in spite of my leaving Substack open. Substack needs to work on that.
Interesting discussion, thanks.
Sadly, me too, a couple prior times.
The link in the first paragraph to download doesn't work.
try it now. I changed the protocol to https. Plain http was blocked by the browser on security grounds
Thanks, I got the skill to download now.