The AI trust crisis 14th December 2023 Dropbox added some new AI features. In the past couple of days these have attracted a firestorm of criticism. Benj Edwards rounds it up in Dropbox spooks users with new AI features that send data to OpenAI when used. The key issue here is that people are worried that their private files on Dropbox are being passed to OpenAI to use as training data for their models—a claim that is strenuously denied by Dropbox. As far as I can tell, Dropbox built some sensible features—summarize on demand, “chat with your data” via Retrieval Augmented Generation—and did a moderately OK job of communicating how they work... but when it comes to data privacy and AI, a “moderately OK job” is a failing grade. Especially if you hold as much of people’s private data as Dropbox does! Two details in particular seem really important. Dropbox have an AI principles document which includes this: Customer trust and the privacy of their data are our foundation. We will not use customer data to train AI models without consent. They also have a checkbox in their settings that looks like this: Update: Some time between me publishing this article and four hours later, that link stopped working. I took that screenshot on my own account. It’s toggled “on”—but I never turned it on myself. Does that mean I’m marked as “consenting” to having my data used to train AI models? I don’t think so: I think this is a combination of confusing wording and the eternal vagueness of what the term “consent” means in a world where everyone agrees to the terms and conditions of everything without reading them. But a LOT of people have come to the conclusion that this means their private data—which they pay Dropbox to protect—is now being funneled into the OpenAI training abyss. People don’t believe OpenAI # Here’s copy from that Dropbox preference box, talking about their “third-party partners”—in this case OpenAI: Your data is never used to train their internal models, and is deleted from third-party servers within 30 days. It’s increasing clear to me like people simply don’t believe OpenAI when they’re told that data won’t be used for training. What’s really going on here is something deeper then: AI is facing a crisis of trust. I quipped on Twitter: “OpenAI are training on every piece of data they see, even when they say they aren’t” is the new “Facebook are showing you ads based on overhearing everything you say through your phone’s microphone” Here’s what I meant by that. Facebook don’t spy on you through your microphone # Have you heard the one about Facebook spying on you through your phone’s microphone and showing you ads based on what you’re talking about? This theory has been floating around for years. From a technical perspective it should be easy to disprove: Mobile phone operating systems don’t allow apps to invisibly access the microphone. Privacy researchers can audit communications between devices and Facebook to confirm if this is happening. Running high quality voice recognition like this at scale is extremely expensive—I had a conversation with a friend who works on server-based machine learning at Apple a few years ago who found the entire idea laughable. The non-technical reasons are even stronger: Facebook say they aren’t doing this. The risk to their reputation if they are caught in a lie is astronomical. As with many conspiracy theories, too many people would have to be “in the loop” and not blow the whistle. Facebook don’t need to do this: there are much, much cheaper and more effective ways to target ads at you than spying through your microphone. These methods have been working incredibly well for years. Facebook gets to show us thousands of ads a year. 99% of those don’t correlate in the slightest to anything we have said out loud. If you keep rolling the dice long enough, eventually a coincidence will strike. Here’s the thing though: none of these arguments matter. If you’ve ever experienced Facebook showing you an ad for something that you were talking about out-loud about moments earlier, you’ve already dismissed everything I just said. You have personally experienced anecdotal evidence which overrides all of my arguments here.

One consistent theme I’ve seen in conversations about this issue is that people are much more comfortable trusting their data to local models that run on their own devices than models hosted in the cloud. The good news is that local models are consistently both increasing in quality and shrinking in size.

Being broadly effective, but somewhat inefficient, is better than being narrowly efficient, but less effective.

Targeting can increase the scale of effects, but this study suggests that the cheaper approach of not targeting so specifically, might actually deliver a greater financial outcome

As Wiberg’s findings point out, the problem with targeting towards conversion optimisation is you are effectively advertising to many people who were already going to buy you.

If I only sell to IT decision-makers, for example, I need some targeting, as I just can’t afford to talk to random consumers. I must pay for some targeting in my media buy, in order to reach a relatively niche audience.  Targeting is no longer a nice to do, but a must have. The interesting question then becomes not should I target, but how can I target effectively?

What they found was any form of second or third-party data led segmenting and targeting of advertising does not outperform a random sample when it comes to accuracy of reaching the actual target.

Contextual ads massively outperform even first party data

We can improve the quality of our targeting much better by just buying ads that appear in the right context, than we can by using my massive first party database to drive the buy, and it’s way cheaper to do that. Putting ads in contextually relevant places beats any form of targeting to individual characteristics. Even using your own data.

The secret to effective, immediate action-based advertising, is perhaps not so much about finding the right people with the right personas and serving them a tailored customised message. It’s to be in the right places. The places where they are already engaging with your category, and then use advertising to make buying easier from that place

Even hard, sales-driving advertising isn’t the tough guy we want it to be. Advertising mostly works when it makes things easier, much more often than when it tries to persuade or invoke a reluctant action.

Thinking about advertising as an ease-making mechanism is much more likely to set us on the right path

If your ad is in the right place, you automatically get the right people, and you also get them at the right time; when they are actually more interested in what you have to sell. You also spend much less to be there than crunching all that data

I sometimes think about the fact that Amazon S3 effectively has to exist until the heat death of the universe. Many millennia from now, our highly-evolved descendants will probably be making use of an equally highly evolved descendant of S3. It is fun to think about how this would be portrayed in science fiction form, where developers pore through change logs and design documents that predate their great-great-great-great grandparents, and users inherit ancient (yet still useful) S3 buckets, curate the content with great care, and then ensure that their progeny will be equally good stewards for all of the precious data stored within

Coming back to the internet analogy, how did Google, Amazon etc ended up so successful? Metcalf’s law explains this. It states that as more users join the network, the value of the network increases thereby attracting even more users. The most important thing here was to make people join your network. The end goal was to build the largest network possible. Google did this with search, Amazon did this with retail, Facebook did this with social.

Collecting as much data as possible is important. But you don’t want just any data. The real competitive advantage lies in having high-quality proprietary data. Think about it this way, what does it take to build an AI system? It takes 1) data, which is the input that goes into the 2) AI models which are analogous to machines and lastly it requires energy to run these models i.e. 3) compute. Today, most AI models have become standardized and are widely available. And on the other hand, the cost of compute is rapidly trending to zero. Hence AI models and compute have become a commodity. The only thing that remains is data. But even data is widely available on the internet. Thus, a company can only have a true competitive advantage when it has access to high-quality proprietary data.

Recently, Chamath Palihapitiya gave an interview where he had this interesting analogy. He compared these large language models like GPT to refrigeration. He said “People that invented refrigeration, made some money. But most of the money was made by Coca-Cola who used refrigeration to build an empire. And so similarly, companies building these large models will make some money, but the Coca-Cola is yet to be built.” What he meant by this is that right now there are lot of companies crawling the open web to scrap the data. Once that is widely available like refrigeration, we will see companies and startups coming up with proprietary data building on top of it

Social networking is much more than just a dominant discourse. We need to go beyond text and images and include its software, interfaces, and networks that depend on a technical infrastructure consisting of offices and their consultants and cleaners, cables and data centers, working in close concert with the movements and habits of the connected billions. Academic internet studies circles have shifted their attention from utopian promises, impulses, and critiques to “mapping” the network’s impact. From digital humanities to data science we see a shift in network-oriented inquiry from Whether and Why, What and Who, to (merely) How. From a sociality of causes to a sociality of net effects. A new generation of humanistic researchers is lured into the “big data” trap, and kept busy capturing user behavior whilst producing seductive eye candy for an image-hungry audience (and vice versa).

We need to politicize the New Electricity, the privately owned utilities of our century, before they disappear into the background.

What remains particularly unexplained is the apparent paradox between the hyper-individualized subject and the herd mentality of the social.

Before we enter the social media sphere, everyone first fills out a profile and choses a username and password in order to create an account. Minutes later, you’re part of the game and you start sharing, creating, playing, as if it has always been like that. The profile is the a priori part and the profiling and targeted advertising cannot operate without it. The platforms present themselves as self-evident. They just are—facilitating our feature-rich lives. Everyone that counts is there. It is through the gate of the profile that we become its subject.

We pull in updates, 24/7, in a real-time global economy of interdependencies, having been taught to read news feeds as interpersonal indicators of the planetary condition

Treating social media as ideology means observing how it binds together media, culture, and identity into an ever-growing cultural performance (and related “cultural studies”) of gender, lifestyle, fashion, brands, celebrity, and news from radio, television, magazines, and the web—all of this imbricated with the entrepreneurial values of venture capital and start-up culture, with their underside of declining livelihoods and growing inequality.

Software, or perhaps more precisely operating systems, offer us an imaginary relationship to our hardware: they do not represent transistors but rather desktops and recycling bins. Software produces users. Without operating system (OS) there would be no access to hardware; without OS no actions, no practices, and thus no user. Each OS, through its advertisements, interpellates a “user”: calls it and offers it a name or image with which to identify. We could say that social media performs the same function, and is even more powerful.

In the age of social media we seem to confess less what we think. It’s considered too risky, too private. We share what we do, and see, in a staged manner. Yes, we share judgments and opinions, but no thoughts. Our Self is too busy for that, always on the move, flexible, open, sporty, sexy, and always ready to connect and express.

Platforms are not stages; they bring together and synthesize (multimedia) data, yes, but what is lacking here is the (curatorial) element of human labor. That’s why there is no media in social media. The platforms operate because of their software, automated procedures, algorithms, and filters, not because of their large staff of editors and designers. Their lack of employees is what makes current debates in terms of racism, anti-Semitism, and jihadism so timely, as social media platforms are currently forced by politicians to employ editors who will have to do the all-too-human monitoring work (filtering out ancient ideologies that refuse to disappear).