In the first approach, the primary interface affordance is an input that directly (for the most part) instructs an AI model(s). In this paradigm, people are authoring prompts that result in text, image, video, etc. generation. These prompts can be sequential, iterative, or un-related. Marquee examples are OpenAI's ChatGPT interface or Midjourney's use of Discord as an input mechanism. Since there are few, if any, UI affordances to guide people these systems need to respond to a very wide range of instructions. Otherwise people get frustrated with their primarily hidden (to the user) limitations.

The second approach doesn't include any UI elements for directly controlling the output of AI models. In other words, there's no input fields for prompt construction. Instead instructions for AI models are created behind the scenes as people go about using application-specific UI elements. People using these systems could be completely unaware an AI model is responsible for the output they see.

The third approach is application specific UI with AI assistance. Here people can construct prompts through a combination of application-specific UI and direct model instructions. These could be additional controls that generate portions of those instructions in the background. Or the ability to directly guide prompt construction through the inclusion or exclusion of content within the application. Examples of this pattern are Microsoft's Copilot suite of products for GitHub, Office, and Windows.

they could be overlays, modals, inline menus and more. What they have in common, however, is that they supplement application specific UIs instead of completely replacing them.

In recent years Sketch’s Mac-ness has become a liability. Requiring every person in a large design organization to use a Mac is not an easy sell. Plus, a new generation of “internet native” users expect different things from their software than old-school Mac connoisseurs: Multiplayer editing, inline commenting, and cloud sync are now table-stakes for any successful creative app.

At the time of Sketch’s launch most UX designers were using Photoshop or Illustrator. Both were expensive and overwrought, and neither were actually created for UX design. Sketch’s innovation wasn’t any particular feature — if anything it was the lack of features. It did a few things really well, and those were exactly the things UX designers wanted. In that way it really embodied the Mac ethos: simple, single-purpose, and fun to use.

Apple pushed hard to attract artists, filmmakers, musicians, and other creative professionals. It started a virtuous cycle. More creatives using Macs meant more potential customers for creative Mac software, which meant more developers started building that software, which in turn attracted even more customers to the platform.And so the Mac ended up with an abundance of improbably-good creative tools. Usually these apps weren’t as feature-rich or powerful as their PC counterparts, but were faster and easier and cheaper and just overall more conducive to the creative process.

Apple is still very interested in selling Macs — precision-milled aluminum computers with custom-designed chips and “XDR” screens. But they no longer care much about The Mac: The operating system, the software platform, its design sensibilities, its unique features, its vibes.

The term-of-art for this style is “skeuomorphism”: modern designs inspired by their antecedents — calculator apps that look like calculators, password-entry fields that look like bank vaults, reminders that look like sticky notes, etc.This skeuomorphic playfulness made downloading a new Mac app delightful. The discomfort of opening a new unfamiliar piece of software was totally offset by the joy of seeing a glossy pixel-perfect rendition of a bookshelf or a bodega or a poker table, complete with surprising little animations.

There are literally dozens of ways to develop cross-platform apps, including Apple’s own Catalyst — but so far, none of these tools can create anything quite as polished as native implementations.So it comes down to user preference: Would you rather have the absolute best app experience, or do you want the ability to use an acceptably-functional app from any of your devices? It seems that users have shifted to prefer the latter.

Unfortunately the appeal of native Mac software was, at its core, driven by brand strategy. Mac users were sold on the idea that they were buying not just a device but an ecosystem, an experience. Apple extended this branding for third-party developers with its yearly Apple Design Awards.

for the first time since the introduction of the original Mac, they’re just computers. Yes, they were technically always “just computers”, but they used to feel like something bigger. Now Macs have become just another way, perhaps the best way, to use Slack or VSCode or Figma or Chrome or Excel.

visionOS’s story diverges from that of the Mac. Apple is no longer a scrappy upstart. Rather, they’re the largest company in the world by market cap. It’s not so much that Apple doesn’t care about indie developers anymore, it’s just that indie developers often end up as the ants crushed beneath Apple’s giant corporate feet.

I think we’ll see a lot of cool indie software for visionOS, but also I think most of it will be small utilities or toys. It takes a lot of effort to build and support apps that people rely on for their productivity or creativity. If even the wildly-popular Mac platform can’t support those kinds of projects anymore, what chance does a luxury headset have?

When content is shorter and maximized for engagement, we often lose track of the origin, history, and context behind it: a new designer is more likely to hear about a UX law from a UX influencer on an Instagram carousel than through the actual research which brought it about.The lack of nuance from algorithm-suggested posts undermines any value we could get from them. For a discipline known for asking "why" and for striving to understand users’ context, it’s time we become more intentional about our own information sources.

Shifts in visual narratives happen every decade or so, so it’s not surprising that the design world is moving away from the corporate flatness of web2. Instead of reminding us of the problems of our current world and the harm that’s been caused by Big Tech, the new, abstract forms of web3 distract us from the crises of the day with the promise of a new virtual world.

the concept of file folders and directories, essential to previous generations’ understanding of computers, is gibberish to many modern students.

Humor us for a moment and picture your favorite neighborhood restaurant. Ours is a corner spot in Fort Greene, Brooklyn. It has overflowing natural light, handmade textile seat cushions, a caramel wood grain throughout, and colorful ornaments dangling from the ceilings. Can you picture yours? Do you feel the warmth and spirit of the place?A Silicon Valley optimizer might say, “Well, they don’t brew their coffee at exactly 200 degrees. And the seats look a little ratty. And the ceiling ornaments don’t serve any function.”But we think that’s exactly the point. That these little, hand-crafted touches give our environment its humanity and spirit. In their absence, we’re left with something universal but utterly sterile — a space that may “perfectly” serve our functional needs, but leave our emotional needs in the lurch.

Operating systems were bubbly and evanescent, like nature. Apps were customizable, in every shape and size. And interfaces drew on real-life metaphors to help you understand them, integrating them effortlessly into your life.But as our everyday software tools and media became global for the first time, the hand of the artist gave way to the whims of the algorithm. And our software became one-size-fits-all in a world full of so many different people. All our opinions, beliefs, and ideas got averaged out — producing the least common denominator: endless sequels that everyone enjoys but no one truly loves.When our software optimizes for numbers alone — no matter the number — it appears doomed to lack a certain spirit, and a certain humanity.

In the end, we decided that we didn’t want to optimize for numbers at all. We wanted to optimize for feelings.While this may seem idealistic at best or naive at worst, the truth is that we already know how to do this. The most profound craftsmanship in our world across art, design, and media has long revolved around feelings.

When Olmstead crafted Central Park, what do you think he was optimizing for? Which metric led to Barry Jenkins’ Moonlight? What data brought the iPhone into this world? The answer is not numerical. It’s all about the feelings, opinions, experiences, and ideas of the maker themself. The great Georgia O’Keefe put it this way: "I have things in my head that are not like what anyone has taught me... so I decided to start anew."

James Turrell took inspiration from astronomy and perceptual psychology. Coco Chanel was most influenced by nuns and religious symbols. David Adjaye drew from Yoruban sculpture, and Steve Jobs from Zen Buddhism and calligraphy.

And yet, in so much modern software today, you’re placed in a drab gray cubicle — anonymized and aggregated until you’re just a daily active user. For minimalism. For simplicity. For scale! But if our hope is to create software with feeling, it means inviting people in to craft it for themselves — to mold it to the contours of their unique lives and taste.

You see — if software is to have soul, it must feel more like the world around it. Which is the biggest clue of all that feeling is what’s missing from today’s software. Because the value of the tools, objects, and artworks that we as humans have surrounded ourselves with for thousands of years goes so far beyond their functionality. In many ways, their primary value might often come from how they make us feel by triggering a memory, helping us carry on a tradition, stimulating our senses, or just creating a moment of peace.This is not to say that metrics should not play a role in what we do. The age of metrics has undeniably led us to some pretty remarkable things! And numbers are a useful measuring stick to keep ourselves honest.But if the religion of technology preaches anything, it celebrates progress and evolution. And so we ask, what comes next? What do we optimize for beyond numbers? How do we bring more of the world around us back into the software in front of us?

For the layman to build their own Folk Interfaces, jigs to wield the media they care about, we must offer simple primitives. A designer in Blender thinks in terms of lighting, camera movements, and materials. An editor in Premiere, in sequences, transitions, titles, and colors. Critically, this is different from automating existing patterns, e.g. making it easy to create a website, simulate the visuals of film photography, or 3D-scan one's room. Instead, it's about building a playground in which those novel computational artifacts can be tinkered with and composed, via a grammar native to their own domain, to produce the fruits of the users' own vision. The goal of the computational tool-maker then is not to teach the layman about recursion, abstraction, or composition, but to provide meaningful primitives (i.e. a system) with which the user can do real work. End-user programming is a red herring: We need to focus on materiality, what some disparage as mere "side effects." The goal is to enable others to feel the agency and power that comes when the world ceases to be immutable.

I hope this mode can paint the picture of software, not as a teleological instrument careening towards automation and ease, but as a medium for intimacy with the matter of our time (images, audio, video), yielding a sense of agency with what, to most, feels like an indelible substrate.