Deleting sections of a bill holding companies accountable for making data-driven decisions that could lead to discrimination in housing, employment, health care, and the like spurred a strong response from civil society organizations including the NAACP, the Japanese American Citizens League, the Autistic Self Advocacy Network, and Asian Americans Advancing Justice, among dozens of others.

In a letter this week to E&C Democrats, obtained by WIRED, the groups wrote: “Privacy rights and civil rights are no longer separate concepts—they are inextricably bound together and must be protected. Abuse of our data is no longer limited to targeted advertising or data breaches. Instead, our data are used in decisions about who gets a mortgage, who gets into which schools, and who gets hired—and who does not.”

these provisions contained generous “pro-business” caveats. For instance, users would be able to opt out of algorithmic decisionmaking only if doing so wasn’t “prohibitively costly” or “demonstrably impracticable due to technological limitations.” Similarly, companies could have limited the public’s knowledge about the results of any audits by simply hiring an independent assessor to complete the task rather than doing so internally.

what if every police patrol car had a bank of cameras that scan not just every number plate but every face within a hundred yards against a national database of outstanding warrants? What if the cameras in the subway do that? All the connected cameras in the city? China is already trying to do this, and we seem to be pretty sure we don’t like that, but why? One could argue that there’s no difference in principle, only in scale, but a change in scale can itself be a change in principle.

Generative AI is now creating a lot of new examples of scale itself as a difference in principle. You could look the emergent abuse of AI image generators, shrug, and talk about Photoshop: there have been fake nudes on the web for as long as there’s been a web. But when high-school boys can load photos of 50 or 500 classmates into an ML model and generate thousands of such images (let’s not even think about video) on a home PC (or their phone), that does seem like an important change. Faking people’s voices has been possible for a long time, but it’s new and different that any idiot can do it themselves. People have always cheated at homework and exams, but the internet made it easy and now ChatGPT makes it (almost) free. Again, something that has always been theoretically possible on a small scale becomes practically possible on a massive scale, and that changes what it means.

This might be a genuinely new and bad thing that we don’t like at all; or, it may be new and we decide we don’t care; we may decide that it’s just a new (worse?) expression of an old thing we don’t worry about; and, it may be that this was indeed being done before, even at scale, but somehow doing it like this makes it different, or just makes us more aware that it’s being done at all. Cambridge Analytica was a hoax, but it catalysed awareness of issues that were real