John Watson looks down at his screen, and we see the message he’s reading on our screen as well. Now, we’re used to seeing extradiegetic text appear on screen with the characters: titles like “Three Years Earlier” or “Lisbon” serve to orient us in a scene. Those titles even can help set the tone of the narrative - think of the snarky humor of the character introduction chyrons on Burn Notice. But this is different: this is capturing the viewer’s screen as part of the narrative itself [1] It’s a remarkably elegant solution from director Paul McGuigan. And it works because we, the viewing audience, have been trained to understand it by the last several years of service-driven, multi-platform, multi-screen applications.

The connection between Sherlock’s intellect and a computer’s becomes more explicit in one of my favorite scenes, later in the episode. Sherlock is called to the scene of the murder from which the episode takes its title.[3] We watch him process the clues from the scene and as he takes them in, that same titling style appears, now employed in a more conventional-seeming expositional mode

But then the shot reverses, and it’s not quite so conventional after all. The titling isn’t just what Sherlock is understanding, it’s what he’s seeing. In the same way that text-message titling can take over our screens because whatever we’re watching TV on is just another screen in a multiplatform computing system, this scene tells us that Sherlock views the whole world through the head-up display of his own genius.

The industrial designers talked about contours that felt gratifying in the hand and actions that provided a fidget-like comfort such as flipping the lid of a Zippo lighter or the satisfying click of a pen.

In video games, the button you press to make a character jump is often a simple binary input (pressed or not), and yet the output combines a very finely-tuned choreography of interactions, animations, sounds, particles, and camera shake to create a rich composition of sensations. The same jump button can feel like a dainty hop or a powerful leap. “Game feel” (a.k.a. “juice”) is the “aesthetic sensation of control” (Steve Swink, Game Feel) you have when playing a game.

The difference comes down to choice—which is to say, Design (with a capital “D”). Game feel is what makes some games feel gratifying to play (a character gliding down a sand dune) and others feel frustrating (sticky jumping, sliding). These decisions become a signature part of a game’s aesthetic feel and gameplay.

The Browser Company has written that software can optimize for emotional needs rather than just functional needs. Jason Yuan has promoted the idea of “fidgetability” where, similar to a key fob or lighter, digital actions can be designed to feel satisfying. Rahul Vohra has talked about making interfaces that are first fun as a toy—enjoyable to use without any greater aim.

The 2D portion is a particle simulation that “feeds” the growing sphere made with Lottie. It’s inspired by the charging animation common in games before your character delivers a big blow. Every action needs a windup. A big action—in order to feel big—needs a big wind up.

This is the big moment—it has to feel gratifying. We again combine 2D and 3D elements. The sphere and checkmark pop in and a massive starburst fills the screen like an enemy hit in Hollow Knight.

Our digital products are trapped behind a hard pane of glass. We use the term “touch”, but we never really touch them. To truly Feel a digital experience and have an app reach through that glass, requires the Designer to employ many redundant techniques. Video games figured this out decades ago. What the screen takes away, you have to add back in: animation, sound, and haptics.

For the layman to build their own Folk Interfaces, jigs to wield the media they care about, we must offer simple primitives. A designer in Blender thinks in terms of lighting, camera movements, and materials. An editor in Premiere, in sequences, transitions, titles, and colors. Critically, this is different from automating existing patterns, e.g. making it easy to create a website, simulate the visuals of film photography, or 3D-scan one's room. Instead, it's about building a playground in which those novel computational artifacts can be tinkered with and composed, via a grammar native to their own domain, to produce the fruits of the users' own vision. The goal of the computational tool-maker then is not to teach the layman about recursion, abstraction, or composition, but to provide meaningful primitives (i.e. a system) with which the user can do real work. End-user programming is a red herring: We need to focus on materiality, what some disparage as mere "side effects." The goal is to enable others to feel the agency and power that comes when the world ceases to be immutable.

I hope this mode can paint the picture of software, not as a teleological instrument careening towards automation and ease, but as a medium for intimacy with the matter of our time (images, audio, video), yielding a sense of agency with what, to most, feels like an indelible substrate.