Interesting article. As I was reading it, I was thinking that some companies deliberately obscure functions behind complicated menus so you don’t do them, such as cancel. The other I was thinking about was advertisements, where would they go? So many UI’s today are designed to maximize advertisement display. Given the prevalence of advertisement supported services. I thought this was an interesting question. Although they could probably just put them on the generic search page you describe.
The WIMP (windows, icon, menus, pointer) metaphor has worked for a long time and I suspect we won’t be saying goodbye anytime soon. The wimp metaphor is a discovery mechanism. You can’t discover purely through language if you don’t know what to ask.
The problem is that the structure of a webpage itself imparts knowledge - people don’t know what they don’t know. A 13-year-old typically has no idea what to do with a voice interface at the Wall Street Journal. That same 13-year-old enjoys deep diving the structure of a Wall Street journal if they have the intellectual curiosity. At some point, you need to be smart enough to use AI and if the goal is to not even need to be smart enough to use the AI, what’s the point. Will people go to school simply to learn language so that they can talk to AI’s? It feels as if reasoning is being outsourced, but how can one use outsourced reasoning if they have not learned to reason?
Did you know that if you hold down the mute button like 10 seconds, it will disappear the cursor in an LG TV? My brother told me that last night. He said he’d told her 10 times.
Why that terrible option exists I don’t know. We knew who to blame though, the guy who watches TV all day.
And if after going to the streaming side you hit the Xfinity Live TV square you’ll be in a world of hurt. The problem is it’s a “purple box” just like “Xfinity Box Set-up”, which is what you always want. My brother said.
Hard for the old people because sometimes the latter is off to the left, off the screen so to speak.
These people have never played a video game in their lives.
The TV rarely seems to understand the voice commands - and no one could enunciate more clearly, and annoyingly than she does - elocution used to be a thing back in her day.
I have to get their trays out there, customized to what they will eat, which has little overlap.
Then I have to determine what they will watch because my brother is paying for Netflix for them. And he’s upset they won’t use it.
“She’s scared” I say.
(To go over to the streaming side - which in its multiplicity is synecdochally referred to as “Netflix” - because the other party gets mad when he sits down to TV at 6 AM and doesn’t immediately find Fox or the business news. He needs it left on that.)
In truth, I’m an odd person to whisper their giant TV (of recent vintage, the brother’s idea) given that my own lives in a closet and involves an antenna and much moving it about, once or twice a year. We actually went to Best Buy once to ask: could we ever stream with this old TV? No.
I’ve done some complicated maneuvers with their smart TV, I’ve gotta say, but I refuse to be the one to turn it off. Because they have 2 remotes, the old and new, and they’ve internalized the firm delusion that it matters which you use for what; in addition to the volume for the Bose that blows up the bass periodically on the rare occasion we enjoy the sound - thank heavens the subtitles seem to be staying fixed across all platforms, what a battle that is on the “old” TV in the back room. (I once spent 15 minutes panic googling how to get the subtitles back to English from Spanish, all because he had said “football” into the remote after we tucked him in; and gotten himself onto Spanish language ESPN. And I didn’t realize that. I was frantically trying to turn people talking Spanish about the NFL - again, didn’t know! Thought they were talking English ‘cuz he hasn’t watched TV with the volume in 15 years - into English subtitles. That’s not a thing, just so you know.)
Even when I finally get into the show that some relative or family member has instructed them to watch – last night it was me actually; I chose comedians and cars, starting with the Michael Richards, Julia L-D, and Obama ones as familiar faces – just manipulating the menu to choose the episode is hard enough.
They’ll never manage a search box. And of course, last night we went away from it for a while and then when we came back, the Netflix suddenly announced we needed to update our household or something. I texted my brother look. I said I don’t think you’re allowed to do this pretending it’s one household business anymore.
I’m not! I’m paying for it for them he said! There followed 10 minutes of signing in, twice - fortunately he had already looked up the cursor problem, which we blamed on the old man who was asleep - but if this happens to you - that way doesn’t work. The QR code is what works. And you’ll need your brother’s wife for the password; she alone knows it.
I often wish they had not lived long enough to see the advent of this technology just as everything else in their lives is getting so exponentially hard.
My mother feels bad that my brother sometimes has to leave work to come over and deal with the effing TV.
By the way, if anyone knows - I separately get her a period drama up on her old iPad when I’m here, because except for Jeopardy she leaves TV to him. This week it’s Little Dorrit (“I didn’t know there was going to be a murder!” she gasped at one point) on free Tubi app. Signed her up so at least it does resume where she left off. But it will not stay on the show. When she returns to it, it’s always back to a menu. Look for Little Dorrit’s face and click on that, I say. I think it’s odd you can’t leave it on the screen.
We are quickly approaching the demise of the graphical user interface and the rise of Star Trek-like direction of software of all kinds. The final flourishes cannot come too soon for me.
I love your ability to imagine the future impacts of a technology — it's something I'm not even bad at, it's just something I somehow never think to do. I'll pushback against one of your predictions, though:
>> "Web sites will wither away. Web design will cease to be an occupation. The user interface will be a generic AI tool, not something that service providers have to think about."
As a business owner, I look forward to being recommended by an AI, but I would be very loathe to stop updating my website. It's a place where I get to control, in minute detail, what potential customers hear about my company. In fact, it's a place to counter the things that other people are saying. And since AI will, for the foreseeable future, scrape the web, making websites is a way to write for the AI.
But good God, you've been thinking about this sort of thing for a lot long than I have! Where might I be going wrong here?
This already exists in rudimentary form on my Roku. Rather than memorize where the closed caption menu controls are for every streaming service, I can simply tell my remote to turn captions on or off.
Last week we went to a restaurant. I wanted to know how late it was open. I went to their website. It seems to me that asking Claude or whatever simply adds an unnecessary complication to an already frictionless process.
As a chemistry prof I used to teach my students MS Excel, thinking they'd be using it for the rest of their lives. But I was wrong. They'll be using AI instead. Nobody needs to learn any software anymore.
I use Excel all the time. I think that trying to explain what I want, especially in some of my more complicated spreadsheets, would be a lot more difficult than simply doing it myself.
Whatever AI produces would have to be proofed, closely and deeply, which again seems like more effort than simply doing it correctly myself.
Similarly, I like having control, and am not eager to cede it.
Finally, many of my spreadsheets contain sensitive data. I am not inclined to invite an outside party -- one controlled by corporate interests -- into the calculations.
True. But that information would have to be entered, by me, in a form the AI could understand, meaning a form I could understand, without involving an outside party.
And, finally an on ramp to the web for people who (for just one example) can’t see well enough to navigate via point and click. The bot agents will obviate the need for screens altogether if they (the content purveyors) get this right.
Microsoft Office has the sort of menu/LLM model you wrote about. The menu structure is cumbersome, but if one wants to do something like create a mail merge, just typing "mail merge" into the search box directly takes one to the menu/feature.
This sounds very much like Doc Searle's concept of an "Intention Economy"
He foresees sellers actively listening for signals from buyers - rather than sellers capturing the eyeballs of "potential" buyers and bombarding "potential" buyers with advertising.
buyers-seller relationships will be controlled in the opposite direction: Customer Relationship Management becomes VENDOR Relationship Management
They will fight against it tooth and nail. Websites will not go quietly; they will sue, they will adopt technological countermeasures to try to stop AI from functioning, they will go to the government and try to get it outlawed, or accuse it of 'hacking' for countering the countermeasures.
The NYT and the WSJ will adapt okay to a world without websites; they have actual customers who pay them money for goods and/or services (i.e. subscription revenue.) But almost everybody else subsists on revenue from horrible ads they're desperate to stop people from blocking. They won't just roll over and be disintermediated.
Interesting article. As I was reading it, I was thinking that some companies deliberately obscure functions behind complicated menus so you don’t do them, such as cancel. The other I was thinking about was advertisements, where would they go? So many UI’s today are designed to maximize advertisement display. Given the prevalence of advertisement supported services. I thought this was an interesting question. Although they could probably just put them on the generic search page you describe.
The WIMP (windows, icon, menus, pointer) metaphor has worked for a long time and I suspect we won’t be saying goodbye anytime soon. The wimp metaphor is a discovery mechanism. You can’t discover purely through language if you don’t know what to ask.
The problem is that the structure of a webpage itself imparts knowledge - people don’t know what they don’t know. A 13-year-old typically has no idea what to do with a voice interface at the Wall Street Journal. That same 13-year-old enjoys deep diving the structure of a Wall Street journal if they have the intellectual curiosity. At some point, you need to be smart enough to use AI and if the goal is to not even need to be smart enough to use the AI, what’s the point. Will people go to school simply to learn language so that they can talk to AI’s? It feels as if reasoning is being outsourced, but how can one use outsourced reasoning if they have not learned to reason?
How does one know the options, like “Letter to the Editor” if they arent made clear? Why wouldn’t ye people running the AI start nerfing them?
Did you know that if you hold down the mute button like 10 seconds, it will disappear the cursor in an LG TV? My brother told me that last night. He said he’d told her 10 times.
Why that terrible option exists I don’t know. We knew who to blame though, the guy who watches TV all day.
And if after going to the streaming side you hit the Xfinity Live TV square you’ll be in a world of hurt. The problem is it’s a “purple box” just like “Xfinity Box Set-up”, which is what you always want. My brother said.
Hard for the old people because sometimes the latter is off to the left, off the screen so to speak.
These people have never played a video game in their lives.
The TV rarely seems to understand the voice commands - and no one could enunciate more clearly, and annoyingly than she does - elocution used to be a thing back in her day.
I have to get their trays out there, customized to what they will eat, which has little overlap.
Then I have to determine what they will watch because my brother is paying for Netflix for them. And he’s upset they won’t use it.
“She’s scared” I say.
(To go over to the streaming side - which in its multiplicity is synecdochally referred to as “Netflix” - because the other party gets mad when he sits down to TV at 6 AM and doesn’t immediately find Fox or the business news. He needs it left on that.)
In truth, I’m an odd person to whisper their giant TV (of recent vintage, the brother’s idea) given that my own lives in a closet and involves an antenna and much moving it about, once or twice a year. We actually went to Best Buy once to ask: could we ever stream with this old TV? No.
I’ve done some complicated maneuvers with their smart TV, I’ve gotta say, but I refuse to be the one to turn it off. Because they have 2 remotes, the old and new, and they’ve internalized the firm delusion that it matters which you use for what; in addition to the volume for the Bose that blows up the bass periodically on the rare occasion we enjoy the sound - thank heavens the subtitles seem to be staying fixed across all platforms, what a battle that is on the “old” TV in the back room. (I once spent 15 minutes panic googling how to get the subtitles back to English from Spanish, all because he had said “football” into the remote after we tucked him in; and gotten himself onto Spanish language ESPN. And I didn’t realize that. I was frantically trying to turn people talking Spanish about the NFL - again, didn’t know! Thought they were talking English ‘cuz he hasn’t watched TV with the volume in 15 years - into English subtitles. That’s not a thing, just so you know.)
Even when I finally get into the show that some relative or family member has instructed them to watch – last night it was me actually; I chose comedians and cars, starting with the Michael Richards, Julia L-D, and Obama ones as familiar faces – just manipulating the menu to choose the episode is hard enough.
They’ll never manage a search box. And of course, last night we went away from it for a while and then when we came back, the Netflix suddenly announced we needed to update our household or something. I texted my brother look. I said I don’t think you’re allowed to do this pretending it’s one household business anymore.
I’m not! I’m paying for it for them he said! There followed 10 minutes of signing in, twice - fortunately he had already looked up the cursor problem, which we blamed on the old man who was asleep - but if this happens to you - that way doesn’t work. The QR code is what works. And you’ll need your brother’s wife for the password; she alone knows it.
I often wish they had not lived long enough to see the advent of this technology just as everything else in their lives is getting so exponentially hard.
My mother feels bad that my brother sometimes has to leave work to come over and deal with the effing TV.
By the way, if anyone knows - I separately get her a period drama up on her old iPad when I’m here, because except for Jeopardy she leaves TV to him. This week it’s Little Dorrit (“I didn’t know there was going to be a murder!” she gasped at one point) on free Tubi app. Signed her up so at least it does resume where she left off. But it will not stay on the show. When she returns to it, it’s always back to a menu. Look for Little Dorrit’s face and click on that, I say. I think it’s odd you can’t leave it on the screen.
We are quickly approaching the demise of the graphical user interface and the rise of Star Trek-like direction of software of all kinds. The final flourishes cannot come too soon for me.
I love your ability to imagine the future impacts of a technology — it's something I'm not even bad at, it's just something I somehow never think to do. I'll pushback against one of your predictions, though:
>> "Web sites will wither away. Web design will cease to be an occupation. The user interface will be a generic AI tool, not something that service providers have to think about."
As a business owner, I look forward to being recommended by an AI, but I would be very loathe to stop updating my website. It's a place where I get to control, in minute detail, what potential customers hear about my company. In fact, it's a place to counter the things that other people are saying. And since AI will, for the foreseeable future, scrape the web, making websites is a way to write for the AI.
But good God, you've been thinking about this sort of thing for a lot long than I have! Where might I be going wrong here?
This already exists in rudimentary form on my Roku. Rather than memorize where the closed caption menu controls are for every streaming service, I can simply tell my remote to turn captions on or off.
It's rare that an internet comment improves my life as much as this just did!
The irony of Substack Live only being available on the phone.
Last week we went to a restaurant. I wanted to know how late it was open. I went to their website. It seems to me that asking Claude or whatever simply adds an unnecessary complication to an already frictionless process.
As a chemistry prof I used to teach my students MS Excel, thinking they'd be using it for the rest of their lives. But I was wrong. They'll be using AI instead. Nobody needs to learn any software anymore.
I use Excel all the time. I think that trying to explain what I want, especially in some of my more complicated spreadsheets, would be a lot more difficult than simply doing it myself.
Whatever AI produces would have to be proofed, closely and deeply, which again seems like more effort than simply doing it correctly myself.
Similarly, I like having control, and am not eager to cede it.
Finally, many of my spreadsheets contain sensitive data. I am not inclined to invite an outside party -- one controlled by corporate interests -- into the calculations.
"What are total payments made to gas stations from Aug 1 to Oct 31, 2022?" That doesn't seem all that hard to say.
True. But that information would have to be entered, by me, in a form the AI could understand, meaning a form I could understand, without involving an outside party.
Not necessarily. Just feed your credit card bills into AI and let it put everything into the right categories.
I agree with you that security and accuracy are valid issues. The VP for going to jail will always be a human.
And, finally an on ramp to the web for people who (for just one example) can’t see well enough to navigate via point and click. The bot agents will obviate the need for screens altogether if they (the content purveyors) get this right.
Will there be a way to keep your text from AI?
Facebook and Amazon have been doing rudimentary adaptive UI for a couple years
Your top menu (eg marketplace vs groups) is ordered based on previous usage.
Microsoft Office has the sort of menu/LLM model you wrote about. The menu structure is cumbersome, but if one wants to do something like create a mail merge, just typing "mail merge" into the search box directly takes one to the menu/feature.
This sounds very much like Doc Searle's concept of an "Intention Economy"
He foresees sellers actively listening for signals from buyers - rather than sellers capturing the eyeballs of "potential" buyers and bombarding "potential" buyers with advertising.
buyers-seller relationships will be controlled in the opposite direction: Customer Relationship Management becomes VENDOR Relationship Management
They will fight against it tooth and nail. Websites will not go quietly; they will sue, they will adopt technological countermeasures to try to stop AI from functioning, they will go to the government and try to get it outlawed, or accuse it of 'hacking' for countering the countermeasures.
The NYT and the WSJ will adapt okay to a world without websites; they have actual customers who pay them money for goods and/or services (i.e. subscription revenue.) But almost everybody else subsists on revenue from horrible ads they're desperate to stop people from blocking. They won't just roll over and be disintermediated.