"I think that A) beliefs about imminent capabilities of AI, and possibilities for the PRC and Russia, will soon freak out much more of the National Security network across the world"
This is already happening, but the freaking is still behind the reality and NatSec types have not fully "priced in" reasonable expectations of future capabilities in terms of their attitudes, planning, urgency, etc. That's not atypical. Usually there is a period of under-freaking following by a panicky overreaction and then easing back into equilibrium especially as new mitigations and control measures prove stable and durable.
But that's because previous changes were more stepwise in nature - big leaps followed by slow and steady improvements or increases in quantities. This time may be different. Things are changing so fast, so much, and perhaps for long enough, that the actual freaking will never actually catch up to the appropriate level of freaking for most establishment types.
O'Malley and Cummings provide two versions of the same message: AI today, albeit interesting and offering glimpses into the future, is not yet close to the AI pinnacle that will render what we see today as no more than tinkertoys for what is coming. Whether it be AGI or not, there is ongoing, rapid advancement of AI. Buckle your seat belts. This is not necessarily an optimistic vision.
I have to concede, O'Malley is much more persuasive than I was. The songs he generated to make his point are great examples of what's currently possible with AI.
I think the point about localized AI being quite limited is important to apply to other realms. When thinking of AI driven small drone warfare, for instance, we can't expect the drones to be running the AI; the battery drain would be immense even if they could cram all the compute needed into the device. As a result the drones will be highly reliant on communicating back to a main AI controller. Not a problem under normal circumstances, but such connections can be relatively easily broken down and overwhelmed by broadcasting noise, creating a zone in while the drones can't act (or at least can't act intelligently.) Jamming communications is annoying for the defenders, too, but it would be catastrophic for drones relying on AI direction from afar.
I think a similar case, though less intentional, holds for having little computers in everything instead of carrying around a device or two with large variety of merely ok functionality. Every device will need a battery and wifi/cellular connection to access the AI, and so carrying a lot of devices will rapidly become tiresome, sort of like carrying a tool box filled with 200 very specific tools vs a toolbox with 10 general use tools and 2-3 specific ones you expect to use now. Especially if the toolbox (phone) can hold 2,000 pretty good general tools at once, people will only want to carry 1-2 specific special use tools outside of it, although those might have a lot of variety by person.
AI controlled drone swarms of the future are different from the “basically a plane” drones we have now, and current externally guided missiles. The primary difference is that plane style drones operate so far away that jamming communications is impractical; you’d have to black out an entire theater to a crazy height. Similarly with guided missiles. Plus, the ordinance those two carry is vastly larger than what small cheap drones can carry; close counts in high capacity warheads.
Small drones are assumed to have to come right up close to attack you. That’s fine, but the nice thing about the AI aspect is that it helps them identify and stay on target that can’t be seen by a third party controller directly. One or the other is fine, but together it means they are vulnerable to losing their “brain” within a smallish area and not being able to acquire targets. You don’t need to black out a huge area because they can’t get you from far away like a laser guided missile from 50,000 ft can, and if they lose their AI recognition they can’t find their targets better than their onboard machine can do while being so small that “close enough” is likely not.
"I think that A) beliefs about imminent capabilities of AI, and possibilities for the PRC and Russia, will soon freak out much more of the National Security network across the world"
This is already happening, but the freaking is still behind the reality and NatSec types have not fully "priced in" reasonable expectations of future capabilities in terms of their attitudes, planning, urgency, etc. That's not atypical. Usually there is a period of under-freaking following by a panicky overreaction and then easing back into equilibrium especially as new mitigations and control measures prove stable and durable.
But that's because previous changes were more stepwise in nature - big leaps followed by slow and steady improvements or increases in quantities. This time may be different. Things are changing so fast, so much, and perhaps for long enough, that the actual freaking will never actually catch up to the appropriate level of freaking for most establishment types.
Or this isn't as big a change as thought. IDK but based on history I'd attach a healthy probability to that possibility.
O'Malley and Cummings provide two versions of the same message: AI today, albeit interesting and offering glimpses into the future, is not yet close to the AI pinnacle that will render what we see today as no more than tinkertoys for what is coming. Whether it be AGI or not, there is ongoing, rapid advancement of AI. Buckle your seat belts. This is not necessarily an optimistic vision.
I suspect that Dominic Cummings’ claims will turn out to have been prescient.
About James O'Malley's idea, I mentioned that idea last March in your Obvious Business Opportunities
for applying LLMs and their relatives post:
https://open.substack.com/pub/arnoldkling/p/obvious-business-opportunities?r=ti5dd&utm_campaign=comment-list-share-cta&utm_medium=web&comments=true&commentId=51282893
I have to concede, O'Malley is much more persuasive than I was. The songs he generated to make his point are great examples of what's currently possible with AI.
I think the point about localized AI being quite limited is important to apply to other realms. When thinking of AI driven small drone warfare, for instance, we can't expect the drones to be running the AI; the battery drain would be immense even if they could cram all the compute needed into the device. As a result the drones will be highly reliant on communicating back to a main AI controller. Not a problem under normal circumstances, but such connections can be relatively easily broken down and overwhelmed by broadcasting noise, creating a zone in while the drones can't act (or at least can't act intelligently.) Jamming communications is annoying for the defenders, too, but it would be catastrophic for drones relying on AI direction from afar.
I think a similar case, though less intentional, holds for having little computers in everything instead of carrying around a device or two with large variety of merely ok functionality. Every device will need a battery and wifi/cellular connection to access the AI, and so carrying a lot of devices will rapidly become tiresome, sort of like carrying a tool box filled with 200 very specific tools vs a toolbox with 10 general use tools and 2-3 specific ones you expect to use now. Especially if the toolbox (phone) can hold 2,000 pretty good general tools at once, people will only want to carry 1-2 specific special use tools outside of it, although those might have a lot of variety by person.
"Jamming communications is annoying for the defenders, too, but it would be catastrophic for drones relying on AI direction from afar."
No more so than for any other piloted drone or externally guided missiles. And I'm pretty sure these have a default plan for if comms are interrupted.
AI controlled drone swarms of the future are different from the “basically a plane” drones we have now, and current externally guided missiles. The primary difference is that plane style drones operate so far away that jamming communications is impractical; you’d have to black out an entire theater to a crazy height. Similarly with guided missiles. Plus, the ordinance those two carry is vastly larger than what small cheap drones can carry; close counts in high capacity warheads.
Small drones are assumed to have to come right up close to attack you. That’s fine, but the nice thing about the AI aspect is that it helps them identify and stay on target that can’t be seen by a third party controller directly. One or the other is fine, but together it means they are vulnerable to losing their “brain” within a smallish area and not being able to acquire targets. You don’t need to black out a huge area because they can’t get you from far away like a laser guided missile from 50,000 ft can, and if they lose their AI recognition they can’t find their targets better than their onboard machine can do while being so small that “close enough” is likely not.