We can take nature’s superpowers and expand them across many more vectors that are interesting to humans: Across scale — far and near, binoculars, zoom, telescope, microscope Across wavelength — UV, IR, heatmaps, nightvision, wifi, magnetic fields, electrical and water currents Across time — view historical imagery, architectural, terrain, geological, and climate changes Across culture — experience the relevance of a place in books, movies, photography, paintings, and language Across space — travel immersively to other locations for tourism, business, and personal connections Across perspective — upside down, inside out, around corners, top down, wider, narrower, out of body Across interpretation — alter the visual and artistic interpretation of your environment, color-shifting, saturation, contrast, sharpness

Headset displays connect sensory extensions directly to your vision. Equipped with sensors that perceive beyond human capabilities, and access to the internet, they can provide information about your surroundings wherever you are. Until now, visual augmentation has been constrained by the tiny display on our phone. By virtue of being integrated with your your eyesight, headsets can open up new kinds of apps that feel more natural. Every app is a superpower. Sensory computing opens up new superpowers that we can borrow from nature. Animals, plants and other organisms can sense things that humans can’t

The first mass-market bicycle for the senses was Apple’s AirPods. Its noise cancellation and transparency mode replace and enhance your hearing. Earbuds are turning into ear computers that will become more easily programmable. This can enable many more kinds of hearing. For example, instantaneous translation may soon be a reality

For the past seven decades, computers have been designed to enhance what your brain can do — think and remember. New kinds of computers will enhance what your senses can do — see, hear, touch, smell, taste. The term spatial computing is emerging to encompass both augmented and virtual reality. I believe we are exploring an even broader paradigm: sensory computing. The phone was a keyhole for peering into this world, and now we’re opening the door.

What happens when put on a headset and open the “Math” app? How could seeing the world through math help you understand both better?

Advances in haptics may open up new kinds of tactile sensations. A kind of second skin, or softwear, if you will. Consider that Apple shipped a feature to help you find lost items that vibrates more strongly as you get closer. What other kinds of data could be translated into haptic feedback?

It may sound far-fetched, but converting olfactory patterns into visual patterns could open up some interesting applications. Perhaps a new kind of cooking experience? Or new medical applications that convert imperceptible scents into visible patterns?

in their efforts to provide better products than their competitors and earn higher prices and margins, suppliers often “overshoot” their market: They give customers more than they need or ultimately are willing to pay for. And more importantly, it means that disruptive technologies that may underperform today, relative to what users in the market demand, may be fully performance-competitive in that same market tomorrow. This was the basis for insisting that the iPhone must have a low-price model: surely Apple would soon run out of new technology to justify the prices it charged for high-end iPhones, and consumers would start buying much cheaper Android phones instead! In fact, as I discussed in after January’s earnings results, the company has gone in the other direction: more devices per customer, higher prices per device, and an increased focus on ongoing revenue from those same customers.

Apple seems to have mostly saturated the high end, slowly adding switchers even as existing iPhone users hold on to their phones longer; what is not happening, though, is what disruption predicts: Apple isn’t losing customers to low-cost competitors for having “overshot” and overpriced its phones. It seems my thesis was right: a superior experience can never be too good — or perhaps I didn’t go far enough.

Jeff Bezos has been writing an annual letter to shareholders since 1997, and he attaches that original letter to one he pens every year. It included this section entitled Obsess Over Customers: From the beginning, our focus has been on offering our customers compelling value. We realized that the Web was, and still is, the World Wide Wait. Therefore, we set out to offer customers something they simply could not get any other way, and began serving them with books. We brought them much more selection than was possible in a physical store (our store would now occupy 6 football fields), and presented it in a useful, easy-to-search, and easy-to-browse format in a store open 365 days a year, 24 hours a day. We maintained a dogged focus on improving the shopping experience, and in 1997 substantially enhanced our store. We now offer customers gift certificates, 1-Click shopping, and vastly more reviews, content, browsing options, and recommendation features. We dramatically lowered prices, further increasing customer value. Word of mouth remains the most powerful customer acquisition tool we have, and we are grateful for the trust our customers have placed in us. Repeat purchases and word of mouth have combined to make Amazon.com the market leader in online bookselling.

This year, after highlighting just how much customers love Amazon (answer: a lot), Bezos wrote: One thing I love about customers is that they are divinely discontent. Their expectations are never static — they go up. It’s human nature. We didn’t ascend from our hunter-gatherer days by being satisfied. People have a voracious appetite for a better way, and yesterday’s ‘wow’ quickly becomes today’s ‘ordinary’. I see that cycle of improvement happening at a faster rate than ever before. It may be because customers have such easy access to more information than ever before — in only a few seconds and with a couple taps on their phones, customers can read reviews, compare prices from multiple retailers, see whether something’s in stock, find out how fast it will ship or be available for pick-up, and more. These examples are from retail, but I sense that the same customer empowerment phenomenon is happening broadly across everything we do at Amazon and most other industries as well. You cannot rest on your laurels in this world. Customers won’t have it.

when it comes to Internet-based services, this customer focus does not come at the expense of a focus on infrastructure or distribution or suppliers: while those were the means to customers in the analog world, in the online world controlling the customer relationship gives a company power over its suppliers, the capital to build out infrastructure, and control over distribution. Bezos is not so much choosing to prioritize customers insomuch as he has unlocked the key to controlling value chains in an era of aggregation.

consumer expectations are not static: they are, as Bezos’ memorably states, “divinely discontent”. What is amazing today is table stakes tomorrow, and, perhaps surprisingly, that makes for a tremendous business opportunity: if your company is predicated on delivering the best possible experience for consumers, then your company will never achieve its goal.

In the case of Amazon, that this unattainable and ever-changing objective is embedded in the company’s culture is, in conjunction with the company’s demonstrated ability to spin up new businesses on the profits of established ones, a sort of perpetual motion machine

Owning the customer relationship by means of delivering a superior experience is how these companies became dominant, and, when they fall, it will be because consumers deserted them, either because the companies lost control of the user experience (a danger for Facebook and Google), or because a paradigm shift made new experiences matter more (a danger for Google and Apple).

For Google Search to increase its profits, it must shift value from web publishers, advertisers and/or users to itself. The only way for Google Search to grow is to make itself worse.

Google’s product managers are each charged with finding ways to increase the profitability of their little corner of the googleverse. That increased profitability can only come from enshittification. Every product manager on Google Search spends their workdays figuring out how to remove a Jenga block. What’s worse, these princelings compete with one another. Their individual progression through the upper echelons of Google’s aristocracy depends as much on others failing as it does on their success. The org chart only has so many VP, SVP and EVP boxes on it, and each layer is much smaller than the previous one. If you’re a VP, every one of your colleagues who makes it to SVP takes a spot that you can no longer get. Those spots are wildly lucrative. Each tier of the hierarchy is worth an order of magnitude more than the tier beneath it. The stakes are so high that they are barely comprehensible. That means that every one of these Jenga-block-pulling execs is playing blind: they don’t — and can’t — coordinate on the ways they’re planning to lower quality in order to improve profits. The exec who decided to save money by reducing the stringency of phone number checking for business accounts didn’t announce this in a company-wide memo. When you’re eating your seed-corn, it’s imperative that you do so behind closed doors, and tell no one what you’ve done. Like any sleight-of-hand artist, you want the audience to see the outcome of the trick (the cost savings), not how it’s done (exposing every searcher in the world to fraud risk to save a buck).

Google/Apple’s mobile duopoly is more cozy than competitive. Google pays Apple $15–20 billion, every single year, to be the default search in Safari and iOS. If Google and Apple were competing over mobile, you’d expect that one of them would drop the sky-high 30 percent rake they charge on in-app payments, but that would mess up their mutual good thing. Instead, these “competitors” charge exactly the same price for a service with minimal operating costs.

your bank, your insurer, your beer company, the companies that make your eyeglasses and your athletic shoes — they’ve all run out of lands to conquer, but instead of weeping, they’re taking it out on you, with worse products that cost more.

Effectively, this is a next-generation step into computational photography. With quad pixels, you can easily see how the iPhone 14 Pro generated 12MP images from a 48MP sensor. It’s grade school division: 48 ÷ 4 = 12. But there is no 24MP sensor in the 15 Pro main camera. The Photonic Engine takes several images from the sensor for each photo, including a 48MP one for detail, and a 12MP quad-pixel one for low-light and noise reduction, then computationally fuses them together to produce a single 24MP image. Apple is having its cake and eating it too, merging the benefits of a sensor with many small pixels with the benefits of a sensor with fewer large pixels. You don’t need to know that this is happening, you just get more detail in your photos from the main camera.

In recent years Sketch’s Mac-ness has become a liability. Requiring every person in a large design organization to use a Mac is not an easy sell. Plus, a new generation of “internet native” users expect different things from their software than old-school Mac connoisseurs: Multiplayer editing, inline commenting, and cloud sync are now table-stakes for any successful creative app.

At the time of Sketch’s launch most UX designers were using Photoshop or Illustrator. Both were expensive and overwrought, and neither were actually created for UX design. Sketch’s innovation wasn’t any particular feature — if anything it was the lack of features. It did a few things really well, and those were exactly the things UX designers wanted. In that way it really embodied the Mac ethos: simple, single-purpose, and fun to use.

Apple pushed hard to attract artists, filmmakers, musicians, and other creative professionals. It started a virtuous cycle. More creatives using Macs meant more potential customers for creative Mac software, which meant more developers started building that software, which in turn attracted even more customers to the platform.And so the Mac ended up with an abundance of improbably-good creative tools. Usually these apps weren’t as feature-rich or powerful as their PC counterparts, but were faster and easier and cheaper and just overall more conducive to the creative process.

Apple is still very interested in selling Macs — precision-milled aluminum computers with custom-designed chips and “XDR” screens. But they no longer care much about The Mac: The operating system, the software platform, its design sensibilities, its unique features, its vibes.

The term-of-art for this style is “skeuomorphism”: modern designs inspired by their antecedents — calculator apps that look like calculators, password-entry fields that look like bank vaults, reminders that look like sticky notes, etc.This skeuomorphic playfulness made downloading a new Mac app delightful. The discomfort of opening a new unfamiliar piece of software was totally offset by the joy of seeing a glossy pixel-perfect rendition of a bookshelf or a bodega or a poker table, complete with surprising little animations.

There are literally dozens of ways to develop cross-platform apps, including Apple’s own Catalyst — but so far, none of these tools can create anything quite as polished as native implementations.So it comes down to user preference: Would you rather have the absolute best app experience, or do you want the ability to use an acceptably-functional app from any of your devices? It seems that users have shifted to prefer the latter.

Unfortunately the appeal of native Mac software was, at its core, driven by brand strategy. Mac users were sold on the idea that they were buying not just a device but an ecosystem, an experience. Apple extended this branding for third-party developers with its yearly Apple Design Awards.

for the first time since the introduction of the original Mac, they’re just computers. Yes, they were technically always “just computers”, but they used to feel like something bigger. Now Macs have become just another way, perhaps the best way, to use Slack or VSCode or Figma or Chrome or Excel.

visionOS’s story diverges from that of the Mac. Apple is no longer a scrappy upstart. Rather, they’re the largest company in the world by market cap. It’s not so much that Apple doesn’t care about indie developers anymore, it’s just that indie developers often end up as the ants crushed beneath Apple’s giant corporate feet.

I think we’ll see a lot of cool indie software for visionOS, but also I think most of it will be small utilities or toys. It takes a lot of effort to build and support apps that people rely on for their productivity or creativity. If even the wildly-popular Mac platform can’t support those kinds of projects anymore, what chance does a luxury headset have?

When I started my career 3G was the hot topic, and every investor kept asking ‘what’s the killer app for 3G?’ It turned out that the killer app for having the internet in your pocket was, well, having the internet in your pocket. But with each of those, we knew what to build next, and with VR we don’t. That tells me that VR has a place in the future. It just doesn’t tell me what kind of place.

The successor to the smartphone will be something that doesn’t just merge AR and VR but make the distinction irrelevant - something that you can wear all day every day, and that can seamlessly both occlude and supplement the real world and generate indistinguishable volumetric space.

Meta, today, has roughly the right price and is working forward to the right device: Apple has started with the right device and will work back to the right price. Meta is trying to catalyse an ecosystem while we wait for the right hardware - Apple is trying to catalyse an ecosystem while we wait for the right price.

one of the things I wondered before the event was how Apple would show a 3D experience in 2D. Meta shows either screenshots from within the system (with the low visual quality inherent in the spec you can make and sell for $500) or shots of someone wearing the headset and grinning - neither are satisfactory. Apple shows the person in the room, with the virtual stuff as though it was really there, because it looks as though it is.

For Meta, the device places you in ‘the metaverse’ and there could be many experiences within that. For Apple, this device itself doesn’t take you anywhere - it’s a screen and there could be five different ‘metaverse’ apps. This iPhone was a piece of glass that could be anything - this is trying to be a piece of glass that can show anything.

A lot of what Apple shows is possibility and experiment - it could be this, this or that, just as when Apple launched the watch it suggested it as fitness, social or fashion, and it turn out to work best for fitness (and is now a huge business).

Mark Zuckerberg, speaking to a Meta all-hands after Apple’s event, made the perfectly reasonable point that Apple hasn’t shown much that no-one had thought of before - there’s no ‘magic’ invention. Everyone already knows we need better screens, eye-tracking and hand-tracking, in a thin and light device.

It’s worth remembering that Meta isn’t in this to make a games device, nor really to sell devices per se - rather, the thesis is that if VR is the next platform, Meta has to make sure it isn’t controlled by a platform owner who can screw them, as Apple did with IDFA in 2021.

On the other hand, the Vision Pro is an argument that current devices just aren’t good enough to break out of the enthusiast and gaming market, incremental improvement isn’t good enough either, and you need a step change in capability.

Apple’s privacy positioning, of course, has new strategic value now that it’s selling a device you wear that’s covered in cameras

the genesis of the current wave of VR was the realisation a decade ago that the VR concepts of the 1990s would work now, and with nothing more than off-the-shelf smartphone components and gaming PCs, plus a bit more work. But ‘a bit more work’ turned out to be thirty or forty billion dollars from Meta and God only knows how much more from Apple - something over $100bn combined, almost certainly.

So it might be that a wearable screen of any kind, no matter how good, is just a staging post - the summit of a foothill on the way to the top of Everest. Maybe the real Reality device is glasses, or contact lenses projecting onto your retina, or some kind of neural connection, all of which might be a decade or decades away again, and the piece of glass in our pocket remains the right device all the way through.

I think the price and the challenge of category creation are tightly connected. Apple has decided that the capabilities of the Vision Pro are the minimum viable product - that it just isn’t worth making or selling a device without a screen so good you can’t see the pixels, pass-through where you can’t see any lag, perfect eye-tracking and perfect hand-tracking. Of course the rest of the industry would like to do that, and will in due course, but Apple has decided you must do that.

For VR, better screens are merely better, but for AR Apple thinks this this level of display system is a base below which you don’t have a product at all.

For Meta, the device places you in ‘the metaverse’ and there could be many experiences within that. For Apple, this device itself doesn’t take you anywhere - it’s a screen and there could be five different ‘metaverse’ apps. The iPhone was a piece of glass that could be anything - this is trying to be a piece of glass that can show anything.

This reminds me a little of when Meta tried to make a phone, and then a Home Screen for a phone, and Mark Zuckerberg said “your phone should be about people.” I thought “no, this is a computer, and there are many apps, some of which are about people and some of which are not.” Indeed there’s also an echo of telco thinking: on a feature phone, ‘internet stuff’ was one or two icons on your portable telephone, but on the iPhone the entire telephone was just one icon on your computer. On a Vision Pro, the ‘Meta Metaverse’ is one app amongst many. You have many apps and panels, which could be 2D or 3D, or could be spaces.

Betting against a first-generation Apple product is a bad trade — from infamous dismissals of the iPhone to disappointment with the original iPad. In fact, this is a reflection of Apple’s strategy: Start with a product that’s more an elegant proof-of-concept than a prime-time hit; rely on early adopters to provide enough runway for its engineers to keep iterating; and trust in unmatched capital, talent, brand equity, and staying power to morph a first-gen toy into a third-gen triumph

We are a long way from making three screens, a glass shield, and an array of supporting hardware light enough to wear for an extended period. Reviewers were (purposefully) allowed to wear the Vision Pro for less than half an hour, and nearly every one said comfort was declining even then. Avatar: The Way of Water is 3 hours and 12 minutes.

Meta’s singular strategic objective is to escape second-tier status and, like Apple and Alphabet, control its distribution. And its path to independence runs through Apple Park. Zuckerberg is spending the GDP of a small country to invent a new world, the metaverse, where Apple doesn’t own the roads or power stations. Vision Pro is insurance against the metaverse evolving into anything more than an incel panic room.

The only product category where VR makes difference is good VR games. Price is not limiting factor, the quality of VR experience is. Beat Saber is good and fun and physical exercise. Half Life: Alyx, is amazing. VR completely supercharges horror games, and scary stalking shooters. Want to fear of your life and get PTSD in the comfort of your home? You can do it. Games can connect people and provide physical exercise. If the 3rd iteration of Vision Pro is good for 2 hours of playing for $2000 Apple will kill the console market. Playstations no more. Apple is not a gaming company, but if Vision Pro becomes better and slightly cheaper, Apple becomes gaming company against its will.

Inspired by biological form and motion, the Dynamic Island is designed to feel like a living organism, with a deliberate elasticity that serves as a playful contrast to the fixed nature of the hardware it embodies.

The products that we build contain complex flows and highly interactive elements. As a result, there's so many important decisions that we need to make. SwiftUI helps by quickly surfacing all of those important details that need your attention, for example, how an image should look when it's loading or how a button appears when it's pressed. These are the types of things that make a product feel complete. They're easily hidden in static design tools but are quickly surfaced when working in a dynamic tool like SwiftUI.That's because SwiftUI makes it easy to build your designs on device. In doing this, you gain a more complete understanding of what you're making. Separate parts now interact together, and you can begin to evaluate the experience as a whole. This process quickly reveals what's working in your design and what still needs attention or polish. On Maps, we've found this to be tremendously helpful.

Apple Vision is technically a VR device that experientially is an AR device, and it’s one of those solutions that, once you have experienced it, is so obviously the correct implementation that it’s hard to believe there was ever any other possible approach to the general concept of computerized glasses.

the Vision is taking that captured image, processing it, and displaying it in front of your eyes in around 4 milliseconds.

Real-time operating systems are used in embedded systems for applications with critical functionality, like a car, for example: it’s ok to have an infotainment system that sometimes hangs or even crashes, in exchange for more flexibility and capability, but the software that actually operates the vehicle has to be reliable and unfailingly fast. This is, in broad strokes, one way to think about how visionOS works: while the user experience is a time-sharing operating system that is indeed a variation of iOS, and runs on the M2 chip, there is a subsystem that primarily operates the R1 chip that is real-time; this means that even if visionOS hangs or crashes, the outside world is still rendered under that magic 12 milliseconds.

I’ll be honest: what this looked like to me was a divorced dad, alone at home with his Vision Pro, perhaps because his wife was irritated at the extent to which he got lost in his own virtual experience.

What can you do with this thing? There’s a good chance that, whatever killer apps may emerge, they don’t need the entire complement of sensors and widgets to deliver a great experience. As that’s discovered, Apple will be able to open a second tier in this category and sell you a simplified model at a lower cost. Meanwhile, the more they manufacture the essentials—high density displays, for example—the higher their yields will become, the more their margins will increase. It takes time to perfect manufacturing processes and build up capacity. Vision Pro isn’t just about 2024’s model. It’s setting up the conditions for Apple to build the next five years of augmented reality wearable technology.

VR/AR doesn’t have to suck ass. It doesn’t have to give you motion sickness. It doesn’t have to use these awkward, stupid controllers you accidentally swap into the wrong hand. It doesn’t have to be fundamentally isolating. If this paradigm shift could have been kicked off by cheap shit, we’d be there already. May as well pursue the other end of the market.

what starts as clunky needn’t remain so. As the technology for augmented reality becomes more affordable, more lightweight, more energy efficient, more stylish, it will be more feasible for more people to use. In the bargain, we’ll get a display technology entirely unshackled from the constraints of a monitor stand. We’ll have much broader canvases subject to the flexibility of digital creativity, collaboration and expression. What this unlocks, we can’t say.

A lot of people see software as a list of features, hardware as a list of specs. But when you think about how much time we spend with these things, maybe they just aren’t that utilitarian. We think of buildings not just as volumes of conditioned air — but also as something architected, as something that can have a profound effect on how you feel, something that can have value in itself (historical buildings and such).

The term “mixed reality” emerged as a way to describe a class of related visual technologies. One of them, virtual reality, seals off users from much of the real world to immerse them in a digital experience. The other, augmented reality, makes digital items appear to inhabit the physical world.

numerous sources say they cannot discern what kind of material Salke and head of television Vernon Sanders want to make. A showrunner with ample experience at the studio says, “There’s no vision for what an Amazon Prime show is. You can’t say, ‘They stand for this kind of storytelling.’ It’s completely random what they make and how they make it.” Another showrunner with multiple series at Amazon finds it baffling that the streamer hasn’t had more success: Amazon has “more money than God,” this person says. “If they wanted to produce unbelievable television, they certainly have the resources to do it.”