When I taught at university I believed that requiring students to write papers on finance and economics allowed them to develop a deeper understanding of the topic. This applied to my own research as well.
Upon reading Arnold’s piece I was led to wonder how the junior partners will develop that depth of understanding if they have relied upon AI to do their research.
The same way a farmer with a tractor doesn't need the same depth of understanding as a farmer with a wooden horse-drawn plow.
The same way the engineer on a diesel-electric locomotive doesn't need the same depth of understanding as the engineer and fireman on a steam locomotive.
The same way a nail gun reduces the depth of understanding needed by a carpenter.
The same way all new technology reshapes its users.
"They won't" is the most likely answer to your question for the vast majority of such junior partners but then you won't need as many such knowledgeable people as senior partners.
Just yesterday I posed the following question to Grok:
"My 2013 Kia Sorrento was sitting parked for 5 hours. When I turned it on, the engine check light came on for the first time and the OBD codes showed P0711 and P0713 code warnings. Later, the light went out without me doing anything to the car. What is the most likely problem with my Kia Sorrento?"
This was Grok's reply- one I had already figured out after about 2 hours of web search and reading:
"The P0711 and P0713 codes both point to issues with the transmission fluid temperature (TFT) sensor circuit in your 2013 Kia Sorento—specifically, range/performance problems (P0711) and high input signals (P0713).
This is a common issue on Sorentos from this era, often triggering on cold starts (like after sitting for several hours) due to erratic sensor readings when the transmission is cold, which aligns with your description.
The check engine light going out on its own suggests an intermittent fault that may resolve as the vehicle warms up, but it could return, especially in colder conditions.The most likely cause is a failing TFT sensor itself, which Kia has addressed in technical service bulletins (TSBs) for similar models and symptoms, recommending replacement of the sensor (often integrated with the internal wiring harness).
Less common but possible related factors include wiring harness damage, poor connections, or even low/contaminated transmission fluid contributing to sensor misreads.
If you notice symptoms like hard or erratic shifting (particularly when cold), that would further support this.I recommend having a mechanic or Kia dealer scan for any recurring codes and inspect/replace the TFT sensor if needed—it's located inside the transmission (requiring pan removal and fluid drain/refill), so it's not a DIY for most people.
I had a similar experience two months ago. My Ram 1500 would not start and the check engine light came on. So I jumped started it and drove to my local AutoZone. AutoZone employee plugged in his diagnostic tool, printed out the error codes, and left me to ponder. I entered the error codes into ChatGPT, and decided to buy a new battery. He sold me a new battery and installed it. I drove home. Error codes were still on. I drove back to AutoZone, borrowed his diagnostic tool and reset/erased the error codes. Went back home and asked ChatGPT whether it was okay for me to have reset the error codes. She said yes, based on the context.
Prior to AI, I would have taken this issue up with the dealer, and probably spent more time and money.
Sometimes an error code leads me to make some adjustments, sometimes not. Either way, if the vehicle stills runs I've always reset the error codes to see if they come back before taking to a shop. I suppose someday there will be an error code that screams not to drive the car but I haven't had that yet.
I'd just purchased my first computer in 1983 and was trying to learn to use a database to convert our paper files at work to electronic format. It seemed to me this might be useful. Our paper files container literally thousands of rate quotes for the intermodal freight company where I was working; looking up specific quotes for a specific customer was tedious and we had an office full of file cabinets. Though conversion was tedious, once completed it was transformative. A rate file could be pulled up instantly for reference, for invoicing, for correspondence. A miracle! I see the same kind of transformation going on now, only at a faster pace. Look out.
There is a paper on how a symbolic maths paper figured out Newtons laws of motion from the observations of planetary motions. I’m not sure how "primed" it was, but it seems like the kind of thing that AIs could become great at. Discovering relationships in complex data that only maths savants can figure out over many years. Like AlphaFold for protein folding.
Large Language Models (LLMs) can make nontrivial contributions to math and physics, if used properly. Separate model instances used to Generate and Verify research steps produce more reliable results than single-shot inference. As a specific example, I describe the use of AI in recent research in quantum field theory (Tomonaga-Schwinger integrability conditions applied to state-dependent modifications of quantum mechanics), work now accepted for publication in Physics Letters B after peer review. Remarkably, the main idea in the paper originated de novo from GPT-5. GPT-5, Gemini, and Qwen-Max were used extensively to perform calculations, find errors, and generate the finished paper.
PDF version
(I intended to post this on arXiv in the cs.AI repository, but what may have been an AI filter informed me it’s not appropriate for cs.AI!)
One of the best ways to develop intuition for LLM behavior and their mastery of physics is to ask the model about a topic (e.g. an already published article) about which the researcher has deep familiarity. In this case I tested several models on their understanding of a 2014 paper I had written on nonlinear QM [7]. In this paper we showed, using a QFT formulation, that state-dependent evolution would cause instantaneous entanglement of widely separated states. This phenomenon is highly problematic and places tight limits on any nonlinear corrections to QM. GPT-5 provided a perfect summary of the paper, which I investigated further as described below.
"Find the slope of the tangent at any point x=a of the function f(x) in the form of a limit? Use this result to find equations of such tangents for a variety of known functions."
"Find the area bound by f(x) and the x-axis between x=a and x=b- generalize this result."
Exactly my point. If you can create those prompts you can actually discover it yourself or already know it. I guess I am asking if an AI today can discover stuff without such specific prompts.
IDK. I think your examples have shown how the possibility is constricted on the "perfect question" end of the spectrum but maybe there is still a sweet spot. And you've got me thinking the hypothetical questioner and AI might do it in an iterative process. ...breaking it down into small pieces...
Something more like, "Explore ways to determine the slope of a tangent to a function f(x)" and "Explore ways to determine areas bound by different functions ". I wish someone with the ability to restrict the training of an AI would publish such a study as I proposed in the top comment. I am smart enough and good enough at math to "discover" calculus given just the idea of the rigorous treatment of limits and the ideas of Riemann summation but here is the thing- I already knew such tools actually existed- I am most certainly not smart enough to self-generate the idea of trying to create them out of those given tools- there lay the stunning brilliance of Newton and Leibniz- can AIs generate that kind of brilliance? I remain unconvinced as of right now.
Like you, Arnold, I think an obvious application of Ai is user interface with computers. The fact that Apple and Microsoft and Google and all the rest have not jumped on this an rode off into the West waving their hats means that it is not yet possible.
My 31-year-old nephew is a fairly recent graduate of law school and got a very lucrative job in corporate law. Has a new baby, wife that doesn't work, and looking to buy a home.
I'm late to the discussion, but: I also very much want AI to take over customer service calls. The current chat systems are awful. But in the case of financial institutions, it's easy to see why they are so cautious. They have to be sure the bot is compliant with all the related laws and regulations, that it can protect customer privacy, and that it can confirm the human's identity. Since AIs are becoming so good at imitating voices, the customer bot has to know that it is not being deceived by another bot. And it must not execute a prompt like: Transfer the balances of all customers in this zip code to my account.
Mir's post, and the idea of AI producing more good stuff than humans, leads to my assertion that better science, military, or business orgs is possible, and likely. But would be shown thru experiments or trials that have better results than the current best human stuff. But we don't even have good measures for what's the best human science. Maybe ai can help get better measures, so that the data we get by measuring becomes more useful.
Personally, I often feel a little bit of >>
“informational deprivation,” a kind of “aversiveness of not knowing,”
<<
so I want to know some details about some things. AI as a trusted ai-pedia is already a thing I and my family often use. But we're not always sure it's always right.
Should we fixate on the volume (large) of machine generated material and the potential noise that comes with it.
But what about when the signal improves? What if the models begin producing output that is better than human, in some sense. What if they begin making scientific discoveries or discovering theorems or even writing poems that are better than those produced by humans? This shifts us into a potential new paradigm on what counts as the primary generator of new ideas.
What does it mean to trust a knowledge system whose internal operations we only barely (if that) understand. What does it mean when human reasoning is not the only reasoning? I suppose if this prediction becomes true we shall find out.
Any comments on why people seem to quote Dwarkesh on things? I spend 30 minutes trying to understand what the heck he’s saying only to find that it’s kind of obvious. I mean, he seems like a nice, earnest guy, and he gets great interviewees. But I’m usually left pretty cold by his insights.
I agree with the notion of natural language interfaces using AI - to interpret a request, convert into a series of API calls, re-state what it is planning to do in straightforward language to get confirmation, then do it.
But I don't have much of a sense of how to train a model to create such an interface on a unique set of systems. The two years/one hour prediction sounds awfully optimistic to me.
I'd be careful arguing that the number of lawyers is going to drop. People predicted the same about bank tellers with the advent of ATMs. Google AI:
"Yes, ATMs reduced the number of tellers per branch, but initially increased overall teller jobs by making branches cheaper to run, leading to more branches and a shift in teller roles toward sales and customer service; however, more recent trends, especially with mobile/online banking, are now causing overall teller positions to decline."
When I taught at university I believed that requiring students to write papers on finance and economics allowed them to develop a deeper understanding of the topic. This applied to my own research as well.
Upon reading Arnold’s piece I was led to wonder how the junior partners will develop that depth of understanding if they have relied upon AI to do their research.
The same way a farmer with a tractor doesn't need the same depth of understanding as a farmer with a wooden horse-drawn plow.
The same way the engineer on a diesel-electric locomotive doesn't need the same depth of understanding as the engineer and fireman on a steam locomotive.
The same way a nail gun reduces the depth of understanding needed by a carpenter.
The same way all new technology reshapes its users.
"They won't" is the most likely answer to your question for the vast majority of such junior partners but then you won't need as many such knowledgeable people as senior partners.
Just yesterday I posed the following question to Grok:
"My 2013 Kia Sorrento was sitting parked for 5 hours. When I turned it on, the engine check light came on for the first time and the OBD codes showed P0711 and P0713 code warnings. Later, the light went out without me doing anything to the car. What is the most likely problem with my Kia Sorrento?"
This was Grok's reply- one I had already figured out after about 2 hours of web search and reading:
"The P0711 and P0713 codes both point to issues with the transmission fluid temperature (TFT) sensor circuit in your 2013 Kia Sorento—specifically, range/performance problems (P0711) and high input signals (P0713).
kia-forums.com +1
This is a common issue on Sorentos from this era, often triggering on cold starts (like after sitting for several hours) due to erratic sensor readings when the transmission is cold, which aligns with your description.
kiasoulforums.com +1
The check engine light going out on its own suggests an intermittent fault that may resolve as the vehicle warms up, but it could return, especially in colder conditions.The most likely cause is a failing TFT sensor itself, which Kia has addressed in technical service bulletins (TSBs) for similar models and symptoms, recommending replacement of the sensor (often integrated with the internal wiring harness).
optimaforums.com +1
Less common but possible related factors include wiring harness damage, poor connections, or even low/contaminated transmission fluid contributing to sensor misreads.
justanswer.com
If you notice symptoms like hard or erratic shifting (particularly when cold), that would further support this.I recommend having a mechanic or Kia dealer scan for any recurring codes and inspect/replace the TFT sensor if needed—it's located inside the transmission (requiring pan removal and fluid drain/refill), so it's not a DIY for most people.
youtube.com
Check if your vehicle qualifies for any extended warranty or TSB coverage, as this has been a frequent repair on 2011-2013 Sorentos. "
The dealership's tech guys first impression was 'Don't have any idea what the problem is, could you leave the car with us for a day or two."
I had a similar experience two months ago. My Ram 1500 would not start and the check engine light came on. So I jumped started it and drove to my local AutoZone. AutoZone employee plugged in his diagnostic tool, printed out the error codes, and left me to ponder. I entered the error codes into ChatGPT, and decided to buy a new battery. He sold me a new battery and installed it. I drove home. Error codes were still on. I drove back to AutoZone, borrowed his diagnostic tool and reset/erased the error codes. Went back home and asked ChatGPT whether it was okay for me to have reset the error codes. She said yes, based on the context.
Prior to AI, I would have taken this issue up with the dealer, and probably spent more time and money.
Truck has been fine since.
Sometimes an error code leads me to make some adjustments, sometimes not. Either way, if the vehicle stills runs I've always reset the error codes to see if they come back before taking to a shop. I suppose someday there will be an error code that screams not to drive the car but I haven't had that yet.
Yes, and it might literally be a scream too.
Mine seem to be somewhat graded already, and perhaps there are some that communicate dire consequences. I just haven’t experienced or looked into it.
I'd just purchased my first computer in 1983 and was trying to learn to use a database to convert our paper files at work to electronic format. It seemed to me this might be useful. Our paper files container literally thousands of rate quotes for the intermodal freight company where I was working; looking up specific quotes for a specific customer was tedious and we had an office full of file cabinets. Though conversion was tedious, once completed it was transformative. A rate file could be pulled up instantly for reference, for invoicing, for correspondence. A miracle! I see the same kind of transformation going on now, only at a faster pace. Look out.
If you trained an AI on only the knowledge base of mathematics as it stood in 1640 would it discover calculus?
If you trained it on Tycho’s data, would it discover the laws of planetary motion?
If you showed it a lot of old TV, would it think people are always in jeopardy from quicksand or hot lava?
There is a paper on how a symbolic maths paper figured out Newtons laws of motion from the observations of planetary motions. I’m not sure how "primed" it was, but it seems like the kind of thing that AIs could become great at. Discovering relationships in complex data that only maths savants can figure out over many years. Like AlphaFold for protein folding.
Another example, from Steve Hsu:
Theoretical Physics with Generative AI
Large Language Models (LLMs) can make nontrivial contributions to math and physics, if used properly. Separate model instances used to Generate and Verify research steps produce more reliable results than single-shot inference. As a specific example, I describe the use of AI in recent research in quantum field theory (Tomonaga-Schwinger integrability conditions applied to state-dependent modifications of quantum mechanics), work now accepted for publication in Physics Letters B after peer review. Remarkably, the main idea in the paper originated de novo from GPT-5. GPT-5, Gemini, and Qwen-Max were used extensively to perform calculations, find errors, and generate the finished paper.
PDF version
(I intended to post this on arXiv in the cs.AI repository, but what may have been an AI filter informed me it’s not appropriate for cs.AI!)
One of the best ways to develop intuition for LLM behavior and their mastery of physics is to ask the model about a topic (e.g. an already published article) about which the researcher has deep familiarity. In this case I tested several models on their understanding of a 2014 paper I had written on nonlinear QM [7]. In this paper we showed, using a QFT formulation, that state-dependent evolution would cause instantaneous entanglement of widely separated states. This phenomenon is highly problematic and places tight limits on any nonlinear corrections to QM. GPT-5 provided a perfect summary of the paper, which I investigated further as described below.
Whatever the limits of AI now or in the future, it seems that "discovering" calculus would be highly dependent on the questions asked.
"Find the slope of the tangent at any point x=a of the function f(x) in the form of a limit? Use this result to find equations of such tangents for a variety of known functions."
"Find the area bound by f(x) and the x-axis between x=a and x=b- generalize this result."
Lol. Seems one or both would be a good bet. Though maybe the questioner would have to know calculus to know to ask those questions?
Exactly my point. If you can create those prompts you can actually discover it yourself or already know it. I guess I am asking if an AI today can discover stuff without such specific prompts.
IDK. I think your examples have shown how the possibility is constricted on the "perfect question" end of the spectrum but maybe there is still a sweet spot. And you've got me thinking the hypothetical questioner and AI might do it in an iterative process. ...breaking it down into small pieces...
Something more like, "Explore ways to determine the slope of a tangent to a function f(x)" and "Explore ways to determine areas bound by different functions ". I wish someone with the ability to restrict the training of an AI would publish such a study as I proposed in the top comment. I am smart enough and good enough at math to "discover" calculus given just the idea of the rigorous treatment of limits and the ideas of Riemann summation but here is the thing- I already knew such tools actually existed- I am most certainly not smart enough to self-generate the idea of trying to create them out of those given tools- there lay the stunning brilliance of Newton and Leibniz- can AIs generate that kind of brilliance? I remain unconvinced as of right now.
Almost certainly not on current AI systems. Perhaps, future AI systems it could.
Like you, Arnold, I think an obvious application of Ai is user interface with computers. The fact that Apple and Microsoft and Google and all the rest have not jumped on this an rode off into the West waving their hats means that it is not yet possible.
Why not use AI? One reason is the simple fact that, for a specific aspect of a particular problem, AI may not perform sufficiently well.
Specific problems benefit from specific tools tailored to the particulars of the problem.
This leads to a bigger question: What is intelligence?
Of course, the answer depends on another question: What problem are you trying to solve?
My 31-year-old nephew is a fairly recent graduate of law school and got a very lucrative job in corporate law. Has a new baby, wife that doesn't work, and looking to buy a home.
He got in just under the wire I suppose.
I'm late to the discussion, but: I also very much want AI to take over customer service calls. The current chat systems are awful. But in the case of financial institutions, it's easy to see why they are so cautious. They have to be sure the bot is compliant with all the related laws and regulations, that it can protect customer privacy, and that it can confirm the human's identity. Since AIs are becoming so good at imitating voices, the customer bot has to know that it is not being deceived by another bot. And it must not execute a prompt like: Transfer the balances of all customers in this zip code to my account.
Mir's post, and the idea of AI producing more good stuff than humans, leads to my assertion that better science, military, or business orgs is possible, and likely. But would be shown thru experiments or trials that have better results than the current best human stuff. But we don't even have good measures for what's the best human science. Maybe ai can help get better measures, so that the data we get by measuring becomes more useful.
Personally, I often feel a little bit of >>
“informational deprivation,” a kind of “aversiveness of not knowing,”
<<
so I want to know some details about some things. AI as a trusted ai-pedia is already a thing I and my family often use. But we're not always sure it's always right.
This is an interesting line of thought
Should we fixate on the volume (large) of machine generated material and the potential noise that comes with it.
But what about when the signal improves? What if the models begin producing output that is better than human, in some sense. What if they begin making scientific discoveries or discovering theorems or even writing poems that are better than those produced by humans? This shifts us into a potential new paradigm on what counts as the primary generator of new ideas.
What does it mean to trust a knowledge system whose internal operations we only barely (if that) understand. What does it mean when human reasoning is not the only reasoning? I suppose if this prediction becomes true we shall find out.
I suspect we won't have to wait for long.
Any comments on why people seem to quote Dwarkesh on things? I spend 30 minutes trying to understand what the heck he’s saying only to find that it’s kind of obvious. I mean, he seems like a nice, earnest guy, and he gets great interviewees. But I’m usually left pretty cold by his insights.
We see AI everywhere but in the productivity statistics.
I agree with the notion of natural language interfaces using AI - to interpret a request, convert into a series of API calls, re-state what it is planning to do in straightforward language to get confirmation, then do it.
But I don't have much of a sense of how to train a model to create such an interface on a unique set of systems. The two years/one hour prediction sounds awfully optimistic to me.
I'd be careful arguing that the number of lawyers is going to drop. People predicted the same about bank tellers with the advent of ATMs. Google AI:
"Yes, ATMs reduced the number of tellers per branch, but initially increased overall teller jobs by making branches cheaper to run, leading to more branches and a shift in teller roles toward sales and customer service; however, more recent trends, especially with mobile/online banking, are now causing overall teller positions to decline."
Sometimes it's this way, sometimes that way.
I saw an article just today that the number of radiologists has increased due to AI interpretation.