Every aspect of interacting with the various institutions that monitored and managed my kids—ACS, the foster care agency, Medicaid clinics—produced new data streams. Diagnoses, whether an appointment was rescheduled, notes on the kids’ appearance and behavior, and my perceived compliance with the clinician’s directives were gathered and circulated through a series of state and municipal data warehouses. And this data was being used as input by machine learning models automating service allocation or claiming to predict the likelihood of child abuse.

The dominant narrative about child welfare is that it is a benevolent system that cares for the most vulnerable. The way data is correlated and named reflects this assumption. But this process of meaning making is highly subjective and contingent. Similar to the term “artificial intelligence,” the altruistic veneer of “child welfare system” is highly effective marketing rather than a description of a concrete set of functions with a mission gone awry.

Child welfare is actually family policing. What AFST presents as the objective determinations of a de-biased system operating above the lowly prejudices of human caseworkers are just technical translations of long-standing convictions about Black pathology. Further, the process of data extraction and analysis produce truths that justify the broader child welfare apparatus of which it is a part.

As the scholar Dorothy Roberts explains in her 2022 book Torn Apart, an astonishing 53 percent of all Black families in the United States have been investigated by family policing agencies.

The kids were contractually the property of New York State and I was just an instrument through which they could supervise their property. In fact, foster parents are the only category of parents legally obligated to open the door to a police officer or a child protective services agent without a warrant. When a foster parent “opens their home” to go through the set of legal processes to become certified to take a foster child, their entire household is subject to policing and surveillance.

Not a single one was surprised about the false allegations. What they were uniformly shocked about was that the kids hadn’t been snatched up. While what happened to us might seem shocking to middle-class readers, for family policing it is the weather. (Black theorist Christina Sharpe describes antiblackness as climate.)

In reality, artificial intelligence encompasses a range of techniques that largely remain useful for a range of uncinematic back-office logistics like processing data from users to better target them with ads, content and product recommendations.

the technologies would at times cause harm, as their humanlike capabilities mean they have the same potential for failure as humans. Among the examples cited: a mistranslation by Facebook’s AI system that rendered “good morning” in Arabic as “hurt them” in English and “attack them” in Hebrew, leading Israeli police to arrest the Palestinian man who posted the greeting, before realizing their error.

Recent local, federal and international regulations and regulatory proposals have sought to address the potential of AI systems to discriminate, manipulate or otherwise cause harm in ways that assume a system is highly competent. They have largely left out the possibility of harm from such AI systems’ simply not working, which is more likely, she says.

Facebook, Twitter and YouTube each took first steps to rein in the worst behavior on their platforms in the heat of the election, but none have confronted how their spaces were structured to become ideal venues for outrage and incitement.

The first step in the process is realizing that the problems we’re experiencing in digital life — how to gather strangers together in public in ways that make it so people generally behave themselves — aren’t new. They’re problems that physical communities have wrestled with for centuries. In physical communities, businesses play a critical role — but so do public libraries, schools, parks and roads. These spaces are often the groundwork that private industry builds itself around: Schools teach and train the next generation of workers; new public parks and plazas often spur private real estate development; businesses transport goods on publicly funded roads; and so on. Public spaces and private industry work symbiotically, if sometimes imperfectly.

These kinds of public spaces mostly don’t exist online. Twitter, Facebook, YouTube and Twitch each offer some aspects of these experiences. But ultimately, they’re all organized around the need for growth and revenue — incentives which are in tension with the critical community functions these institutions also serve, and with the heavy staffing models they require.

Recent peer-reviewed research from three professors at the University of Virginia demonstrates how dramatically the design of platforms can affect how people behave on them. In their study, in months where conservative-leaning users visited Facebook more, they saw much more ideological content than normal, whereas in months where they visited Reddit more they “read news that was 50 percent more moderate than what they typically read.” (This effect was smaller but similar for political liberals). Same people, different platforms, and dramatically different news diets as a result.

Wikipedia is probably the best-known example of this kind of institution — a nonprofit, mission-driven piece of digital infrastructure. The nonprofit Internet Archive, which bills itself as a free “digital library,” a repository of books, movies and music and over 500 billion archived webpages to create a living history of the internet, is another. But what we need are not just information services with a mission-driven agenda, but spaces where people can talk, share and relate without those relationships being distorted and shaped by profit-seeking incentive structures.

Users can post only once a day, every post is read by a moderating team, and if you’re too salty or run afoul of other norms, you’re encouraged to rewrite. This is terrible for short-term engagement — flame wars drive attention and use, after all — and as a business model, all those moderators are costly. But there’s a long-term payoff: two-thirds of Vermont households are on the Forum, and many Vermonters find it a valuable place for thoughtful public discussions.

In fact, public digital infrastructures might be the right place to start exploring how to reinvent governance and civil society more broadly.

If mission, design and governance are important ingredients, the final component is what might be called digital essential workers — professionals like librarians whose job is to manage, steward, and care for the people in these spaces. This care work is one of the pillars of successful physical communities, which has been abstracted away by the existing tech platforms. S

The truth is that Facebook, Google and Twitter have displaced and sucked the revenue out of an entire ecosystem of local journalistic enterprises and other institutions that served some of these public functions.

Whoever is behind the scenes of these movies fails to understand that you can actually make more money by making something that is "good". That making a "good" movie means people will want to watch your movie multiple times and then purchase it again later, and purchase more tickets, and purchase the streaming service with your movie, and purchase the box-set with your movie.

There are three ways you could make a film like this, and make it somewhat "good", whatever that term may mean. You could:A) Make a good movie. Make a movie that is artistically fulfilling, filled to the brim with interesting themes and ideas, passionate craftwork, talented artistry. Something that makes people proud that they went to the movies.B) Make a good Jurassic Park/World movie. Make something that builds on the source material before it in an interesting way and gives itself a purpose to exist in the first place outside of being just a cash cow. Something that makes people proud to be a fan of the franchise.C) Make a good dinosaur movie. Just show a bunch of fucking dinosaurs crashing into each other. Something stupid but ultimately fun. Something that makes people proud to have eyes and ears so that they can see something cool.

The impact is particularly noticeable among Generation Z and millennials, the first generations to come of age with the possibility of knowing where their peers are at all times. It has changed how friends communicate with one another and blurred lines of privacy. Friends now, sometimes unwittingly yet obsessively, check one another’s locations and bypass whole conversations — about where somebody is, what they are doing or how their days are going — when socializing. All of that information can be gleaned from Find My.

Although Find My is not marketed as a social experience, sharing locations has become a test of sorts, much like being included on a close friends list on Instagram or on a private story on Snapchat can signal closer friendships.

With Find My, “you aren’t actively choosing to do something as you reach a certain location because you’re constantly sharing your location,” said Michael Saker, a senior lecturer in digital sociology at City, University of London. As a result, “there’s an intimacy that’s intertwined with that act,” he added. “There’s a verification of being friends.”

Good thinking doesn’t happen by passive osmosis of other people’s good thinking. You have to actually write essays and journals to debug yourself and your ideas. And then, crucially, act (a) on this thinking.

Consider a university professor teaching a course. Does she say anything original? Do you think she should cancel her course because somebody else discovered the things she wants to teach? Or does she have to cancel her course simply because there is a similar course at some other university? Or consider yourself. Do you avoid having conversations with your friends when you think you have nothing original to say? Do you share things with them? Do you give advice? Do you help to understand things?

We’ve put the web in the hands of nearly everyone in society, but we’ve spent a depressingly small amount of time helping folks use the web as a building material.

In other words, when Kylie Jenner posts a petition demanding that Meta “Make Instagram Instagram again”, the honest answer is that changing Instagram is the most Instagram-like behavior possible.

The first trend is the shift towards ever more immersive mediums. Facebook, for example, started with text but exploded with the addition of photos. Instagram started with photos and expanded into video. Gaming was the first to make this progression, and is well into the 3D era. The next step is full immersion — virtual reality — and while the format has yet to penetrate the mainstream this progression in mediums is perhaps the most obvious reason to be bullish about the possibility.

The second trend is the increase in artificial intelligence. I’m using the term colloquially to refer to the overall trend of computers getting smarter and more useful, even if those smarts are a function of simple algorithms, machine learning, or, perhaps someday, something approaching general intelligence.

The third trend is the change in interaction models from user-directed to computer-controlled. The first version of Facebook relied on users clicking on links to visit different profiles; the News Feed changed the interaction model to scrolling. Stories reduced that to tapping, and Reels/TikTok is about swiping. YouTube has gone further than anyone here: Autoplay simply plays the next video without any interaction required at all.

The problem is that the main incentive for pursuing corporate sustainability work is its ability to create more marketable products. Outside of sustainability efforts that are pursued based on compliance or regulation, much of this work is built around the idea of pandering to consumers' understanding of sustainability, something that’s notoriously variable.

in Instagram apologies, even when someone ostensibly confronts their ugliness, it’s hard to read the gesture as anything but an effort to publicly reclaim their image. But at least the Notes App Apology permitted us a semblance of sincerity, and suggested there might be a human being who typed the message—even if that human was an intern or assistant. There’s nothing sincere about a trickle-down excuse crafted to look pretty for Instagram grids, and the processed nature of Photoshopped Apologies implies the absence of the one thing all genuine apologies must possess: accountability straight from the person who committed the transgression.

Advertisers don’t really want to know who you are - they want to show diaper ads to people who have babies, not to show them to people who don’t, and to have some sense of which ads drove half a million sales and which ads drove a million sales.

In practice, ‘showing car ads to people who read about cars’ led the adtech industry to build vast piles of semi-random personal data, aggregated, disaggregated, traded, passed around and sometimes just lost, partly because it could and partly because that appeared to be the only way to do it. After half a decade of backlash, there are now a bunch of projects trying to get to the same underlying advertiser aims - to show ads that are relevant, and get some measure of ad effectiveness - while keeping the private data private.

Apple has pursued a very clear theory that analysis and tracking is private if it happens on your device and is not private if leaves your device or happens in the cloud. Hence, it’s built a complex system of tracking and analysis on your iPhone, but is adamant that this is private because the data stays on the device. People have seemed to accept this (so far - or perhaps the just haven’t noticed it), but acting on the same theory Apple also created a CSAM scanning system that it thought was entirely private - ‘it only happens your device!’ - that created a huge privacy backlash, because a bunch of other people think that if your phone is scanning your photos, that isn’t ‘private’ at all. So is ‘on device’ private or not? What’s the rule? What if Apple tried the same model for ‘private’ ads in Safari? How will the public take FLoC? I don’t think we know.

On / off device is one test, but another and much broader is first party / third party: the idea it’s OK for a website to track what you do on that website but not OK for adtech companies to track you across many different websites. This is the core of the cookie question

At this point one answer is to cut across all these questions and say that what really matters is whether you disclose whatever you’re doing and get consent. Steve Jobs liked this argument. But in practice, as we've discovered, ‘get consent’ means endless cookie pop-ups full of endless incomprehensible questions that no normal consumer should be expected to understand, and that just train people to click ‘stop bothering me’. Meanwhile, Apple’s on-device tracking doesn't ask for permission, and opts you in by default, because, of course, Apple thinks that if it's on the device it's private. Perhaps ‘consent’ is not a complete solution after all.

If you can only analyse behaviour within one site but not across many sites, or make it much harder to do that, companies that have a big site where people spend lots of time have better targeting information and make more money from advertising. If you can only track behaviour across lots of different sites if you do it ‘privately’ on the device or in the browser, then the companies that control the device or the browser have much more control over that advertising

it is very difficult to figure out what specific effect ATT has because there are so many factors involved

If ATT were so significantly kneecapping revenue, I would think we would see a pronounced skew against North America compared to elsewhere. But that is not the case. Revenue in North America is only slightly off compared to the company total, and it is increasing how much it earns per North American user compared to the rest of the world.

iOS is far more popular in the U.S. and Canada than it is in Europe, but Meta incurred a greater revenue decline — in absolute terms and, especially, in percentage terms — in Europe. Meta was still posting year-over-year gains in both those regions until this most recent quarter, even though ATT rolled out over a year ago.

there are those who believe highly-targeted advertisements are a fair trade-off because they offer businesses a more accurate means of finding their customers, and the behavioural data collected from all of us is valuable only in the aggregate. That is, as I understand it, the view of analysts like Seufert, Benedict Evans, and Ben Thompson. Frequent readers will not be surprised to know I disagree with this premise. Regardless of how many user agreements we sign and privacy policies we read, we cannot know the full extent of the data economy. Personal information about us is being collected, shared, combined, and repackaged. It may only be profitable in aggregate, but it is useful with finer granularity, so it is unsurprising that it is indefinitely warehoused in detail.

Seufert asked, rhetorically, “what happens when ads aren’t personalized?”, answering “digital ads resemble TV ads: jarring distractions from core content experience. Non-personalized is another way of saying irrelevant, or at best, randomly relevant.”

does it make sense to build the internet’s economy on the backs of a few hundred brokers none of us have heard of, trading and merging our personal information in the hope of generating a slightly better click-through rate?

Then there is the much bigger question of whether people should even be able to opt into such widespread tracking. We simply cannot be informed consumers in every aspect of our lives, and we cannot foresee how this information will be used and abused in the full extent of time. It sounds boring, but what is so wrong with requiring data minimization at every turn, permitting only the most relevant personal data to be collected, and restricting the ability for this information to be shared or combined?

Does ATT really “[deprive] consumers of widespread ad relevancy and advertisers and publishers of commercial opportunity”? Even if it does — which I doubt — has that commercial opportunity really existed with meaningful consumer awareness and choice? Or is this entire market illegitimate, artificially inflated by our inability to avoid becoming its subjects?

Conflicts like these are one of many reasons why privacy rights should be established by regulators, not individual companies. Privacy must not be a luxury good, or something you opt into, and it should not be a radical position to say so. We all value different degrees of privacy, but it should not be possible for businesses to be built on whether we have rights at all. The digital economy should not be built on such rickety and obviously flawed foundations.

“I’ve been on the internet for the last 10 years and I don’t know if I like what I like or what an algorithm wants me to like,” Peter wrote. She’d come to see social networks’ algorithmic recommendations as a kind of psychic intrusion, surreptitiously reshaping what she’s shown online and, thus, her understanding of her own inclinations and tastes.

Besieged by automated recommendations, we are left to guess exactly how they are influencing us, feeling in some moments misperceived or misled and in other moments clocked with eerie precision.

I care about people more than I care about positions or beliefs, which I tend to consider a subservient class of psychological phenomena. That is to say: I think people wear beliefs like clothes; they wear what they have grown to consider sensible or attractive; they wear what they feel flatters them; they wear what keeps them dry and warm in inclement winter. They believe their opinions, tastes, philosophies are who they are, but they are mistaken. (Aging is largely learning what one is not, it seems to me).

Criticism must serve some function to justify the pain it causes: it must, say, avert a disastrous course of action being deliberated by a group, or help thwart the rise of a barbarous politician. But this rarely occurs. Most criticism, even of the most erudite sort, is, as we all know, wasted breath: preached to one’s own choir, comically or indignantly cruel to those one doesn’t respect, unlikely to change the behavior of anyone not already in agreement.On the other hand! There persists the idea that culture arises out of the scrum of colliding perspectives, and that it is therefore a moral duty to remonstrate against stupidity, performative emoting, deceitful art, destructively banal fiction, and so on. If one doesn’t speak up, one cannot lament the triumph of moral and imaginative vacuity.

One must believe, of course, that there are abstractions worth protecting, and therefore abstractions worth hurting others for, in order to criticize; and the endless repetitiveness of cultural history seems to devalue such abstractions as surely as bad art and cliche devalue words.

Material-based software can also have gentler learning curves, because the user only needs to learn a small set of rules about how different metaphors in the software interact with each other rather than learning deep hierarchies of menus and terminologies. In the best case, users can continue to discover new capabilities and workflows long after the initial release of the software.

Take nonlinear video editing software, like Final Cut Pro and Premiere Pro, for example. Though they have their fair share of menu complexity, it’s pretty intuitive for anyone to understand the basic building blocks of video and audio clips sitting on a layered timeline. Here, the video and audio clips are composed of a software material that have their own laws of physics: they can grow and shrink to take up more or less time on the timeline. They can be moved around and layered in front of or behind other clips, but they can’t occupy the same space on the same “track” of the timeline. The timeline that runs left-to-right is a kind of “world” in which the materials of audio and video exist.

In complexity science, ‘emergence’ describes the way that interactions between individual components in a complex system can give rise to new behavior, patterns, or qualities. For example, the quality of ‘wetness’ cannot be found in a single water molecule, but instead arises from the interaction of many water molecules together. In living systems, emergence is at the core of adaptive evolution.

Design for emergence prioritizes open-ended combinatorial possibilities such that the design object can be composed and adapted to a wide variety of contextual and idiosyncratic niches by its end-user. LEGO offers an example — a simple set of blocks with a shared protocol for connecting to one another from which a nearly infinite array of forms can emerge. Yet as we will see, design for emergence can generate value well beyond children’s toys.

In contrast to high modern design, user-centered design takes a more modest position; the designer does not inherently know everything, and therefore she must meticulously study the needs and behaviors of users in order to produce a good design. User-centered design remains the dominant design paradigm today, employed by environmental designers, tech companies, and design agencies around the world.

In this paradigm, design is about gaining knowledge from the user, identifying desirable outcomes, and controlling as much of the process as possible to achieve those outcomes. ‘Design’ remains synonymous with maximizing control.

But consider even the ‘desire path’ example pictured above. The modal user may be well supported by paving the desire path indicated by their behavior, but what good is a paved path leading to stairs for a wheelchair user? In practice, user-centered design tends to privilege the modal user at the expense of the long-tail user whose needs may be just as great.

User-centered design has a better track record than high modern design, but it still exerts a homogenizing effect. The needs of the modal user are accommodated and scaled through software or industrial manufacturing, while power users and those with edge cases can do nothing but actively petition the designer for attention. In most cases, diverse users with a wide variety of niche use cases are forced to conform to the behavior of the modal user.

In design for emergence, the designer assumes that the end-user holds relevant knowledge and gives them extensive control over the design. Rather than designing the end result, we design the user’s experience of designing their own end result. In this way we can think of design for emergence as a form of ‘meta-design.’

In other words, to address the long-tail problem, the tool must be flexible enough that it can be adapted to unexpected and idiosyncratic problem spaces—especially those unanticipated by the tool’s designer.

In contrast to user-centered design, design for emergence invites the user into the design process not only as a subject of study, but as a collaborator with agency and control.

What all these tools have in common is support for open-ended adaptation to highly contextual problems without the need for technical knowledge. Rather than building a static, purpose-built solution to a single common problem with lots of users (and lots of competitors), they’ve won robust user bases by supporting a broad swath of long-tail user needs.

Design for emergence is composable. It provides a limited ‘alphabet’ and a generative grammar that’s easy to learn and employ, yet can be extended to create powerful, complex applications. As Seymour Papert once remarked, “English is a language for children,” but this fact, “does not preclude its being also a language for poets, scientists, and philosophers.”

Subject after subject reported that they were on Tinder to find someone to love and to love them back and defined love in the most traditional of terms: something that took work, a container in which sex was sacred and where intimacy built over time. They acknowledged that their encounters on Tinder didn’t offer that, yet they went to Tinder to find it. The contradiction was confusing: They wanted sex to be meaningful but felt that Tinder removed the sacredness. They wanted bonds to be lasting but acknowledged they were easily broken.