Agents streaming UI is starting to happen

Happy Friday!

So UX-y people have always had the instinct to design not so much the screens, but the overall interaction elements, rules, including customer understanding, task analysis and so on.

Think about Nicholas Negroponte's Architecture Machine Group at MIT (late 1960s–70s, the lab that became the Media Lab) was explicitly about machines that design with and for you, adapting to the person rather than the person adapting to the machine.

Or think about Allan Kay and the mallable interface. A lot of object oriented stuff came out of this, but it ended up in engineering, not in UX.

With AI bringing some smarts, UI on demand might actually work now.

Christine Vallaure wrote about a new protocol from Google in UXdesign. Basically, if the interface is generated fresh for each request and then thrown away, the unit of design stops being the screen and becomes the system that decides what screen to make — which is context design, not screen design.

The way this one works:

1. Your agent (Claude, Gemini, etc.) sends some JSON to the client.
2. The client renders that using native UI however it wants.
3. The user interacts with the UI.
4. The output is sent back to the agent.

There are other technical approaches out there, but it’s happening. Or at least there are large companies pushing for generative UI to finally work. I can see this working out, mainly because the models are smart enough.

So my question is: what do UX people do about this? I am thinking:

1. Experiment with running these,
2. and experiment with providing context to the agent about the users and tasks and UX (this is the cool bit),
3. and do a lot of testing (ok, evals).

These things take off if they have apps that support them, with a little luck Google will support theirs, and Anthropic has MCP which also has a UI component.

Finally, Josh Clark and Veronika Kindred wrote Sentient Design partially about this, worth checking out.

👀 Interesting this week.

AI is a Train, not a Bicycle: this one I found really fascinating:

GPS was different. The ethnographers put it bluntly: “For the first time in history the navigator can completely rely on technology and travel successfully knowing nothing about navigation and very little about the environment.”

We are using the wrong words for AI: (intelligence, hallucination, AGI, consciousness, agent)

Medicine used ‘‘hallucination’ first to describe sensory errors in the brain. Machine learning borrowed it later, for structural errors in algorithms. There is a tension now from people who want to lock the old meaning down, as though technical precision were the only valid way to see language.

While everyone talks about AI, design is gaining power, UX Collective. Design is quietly being promoted to a strategic function: Microsoft named its first-ever Chief Design Officer, Samsung hired Mauro Porcini, and Airbnb's Joe Gebbia took a design-leadership role in the US government.

Related to UI: Designing With Uncertainty: How AI Supercharges Probabilistic Thinking, Smashing Magazine. AI is "probabilistic systems wrapped in deterministic interfaces".

Dual Translation. Totally fits with my experience.

The autonomy dial: a pattern toolkit for designing human control over AI.

Judgment, Taste, and the Parts of Research We Struggle to Define. Saeideh Bakhshi: judgment is how you choose with too little information, taste is how you choose with too much. Ouch!