AI Sucks

US residents are being urged to discuss their mental health conditions and personal beliefs with chatbots, and their simplest and best-known options are platforms whose owners are cozy with the Trump administration.

Chatbots, likewise, escalate the risks of typical online secret-sharing. Their conversational design can draw out private information in a format that can be more vivid and revealing — and, if exposed, embarrassing — than even something like a Google search.

Like the NSA’s anti-terrorism programs, the data-sharing could be framed in wholesome, prosocial ways. A 14-year-old wonders if they might be transgender, or a woman seeks support for an abortion? Of course OpenAI would help flag that — they’re just protecting children.

If AI companies are genuinely dedicated to building trustworthy services for therapy, they could commit to raising the privacy and security bar for bots that people use to discuss sensitive topics. They could focus on meeting compliance standards for the Health Insurance Portability and Accountability Act (HIPAA) or on designing systems whose logs are encrypted in a way that they can’t access, so there’s nothing to turn over. But whatever they do right now, it’s undercut by their ongoing support for an administration that holds contempt for the civil liberties people rely on to freely share their thoughts, including with a chatbot.

The obvious takeaway from this is “don’t get therapy from a chatbot, especially not from a high-profile platform, especially if you’re in the US, especially not right now.” The more important takeaway is that if chatbot makers are going to ask users to divulge their greatest vulnerabilities, they should do so with the kinds of privacy protections medical professionals are required to adhere to, in a world where the government seems likely to respect that privacy.

The Danish welfare authority, Udbetaling Danmark (UDK), risks discriminating against people with disabilities, low-income individuals, migrants, refugees, and marginalized racial groups through its use of artificial intelligence (AI) tools to flag individuals for social benefits fraud investigations, Amnesty International said today in a new report.

To power these fraud-detection algorithms, Danish authorities have enacted laws that enable extensive collection and merging of personal data from public databases of millions of Danish residents.  The data includes information on residency status and movements, citizenship, place of birth, and family relationships — sensitive data points that can also serve as proxies for a person’s race, ethnicity, or sexual orientation

Five days a week, eight hours a day, Abrha sat in the Sama warehouse in Nairobi, moderating content from the very conflict he had escaped — even sometimes a bombing from his hometown. Each day brought a deluge of hate speech directed at Tigrayans, and dread that the next dead body might be his father, the next rape victim his sister.

The horror of his work reached a devastating peak when Abrha came across his cousin’s body while moderating content. It was a brutal reminder of the very real and personal stakes of the conflict he was being forced to witness daily through a computer screen.

When the company offered to promote him to content moderator with a slight pay increase, he jumped at the opportunity, unaware of the implications of the decision. Kings soon found himself confronting content that haunted him day and night. The worst was what they coded as CSAM, or child sexual abuse material. Day after day, he sifted through texts, pictures and videos vividly depicting the violation of children.

When Ranta’s sister died, she said her boss gave her a few days off but wouldn’t let her switch to less traumatic content streams when she returned to moderating content — even though there was an opening.

She then learned that the company had stopped making health insurance contributions shortly after she started working, despite continued deductions from her paycheck. Now she was saddled with bills she couldn’t afford to pay.

Every day, content moderators are forced to confront the darkest corners of humanity. They wade through a toxic swamp of violence, hate speech, sexual abuse and graphic imagery.

Many moderators report symptoms of post-traumatic stress and vicarious trauma: nightmares, flashbacks and severe anxiety are common. Some develop a deep-seated mistrust of the world around them, forever changed by the constant exposure to human cruelty. As one worker told me, “I came into this job believing in the basic goodness of people. Now, I’m not sure I believe in anything anymore. If people can do this, then what’s there to believe?”

AI is far more energy-intensive on data centers than typical cloud-based applications. According to Goldman Sachs, a ChatGPT query needs nearly 10 times as much electricity to process as a Google search, and data center power demand will grow 160% by 2030. Goldman competitor Morgan Stanley’s research has made similar findings, projecting data center emissions globally to accumulate to 2.5bn metric tons of CO2 equivalent by 2030.

In the meantime, all five tech companies have claimed carbon neutrality, though Google dropped the label last year as it stepped up its carbon accounting standards. Amazon is the most recent company to do so, claiming in July that it met its goal seven years early, and that it had implemented a gross emissions cut of 3%.

The idea that AI’s climate benefits will outpace its environmental costs is largely speculative, however, especially given that generative-AI tools are themselves tremendously resource-hungry. Within the next six years, the data centers required to develop and run the kinds of next-generation AI models that Microsoft is investing in may use more power than all of India. They will be cooled by millions upon millions of gallons of water.

Microsoft isn’t a company that exists to fight climate change, and it doesn’t have to assume responsibility for saving our planet. Yet the company is trying to convince the public that by investing in a technology that is also being used to enrich fossil-fuel companies, society will be better equipped to resolve the environmental crisis. Some of the company’s own employees described this idea to me as ridiculous.

Microsoft’s own climate commitments in 2020. These were made during a moment of heightened climate activism; millions around the world, including tech workers, had just rallied to protest the lack of coordinated action to cut back carbon emissions.Microsoft has failed to reduce its annual emissions each year since then. Its latest environmental report, released this May, shows a 29 percent increase in emissions since 2020—a change that has been driven in no small part by recent AI development, as the company explains in the report. “All of Microsoft’s public statements and publications paint a beautiful picture of the uses of AI for sustainability,” Alpine told me. “But this focus on the positives is hiding the whole story, which is much darker.”

Microsoft is reportedly planning a $100 billion supercomputer to support the next generations of OpenAI’s technologies; it could require as much energy annually as 4 million American homes. Abandoning all of this would be like the U.S. outlawing cars after designing its entire highway system around them.

In 2023, Google and Microsoft data centers each consumed more electricity than is consumed by over 100 countries.

And there is no sign of this energy consumption slowing down. NVIDIA has estimated it would sell 3.5 million of their newest GPU, which would consume electricity equivalent to 1 million US households.

Google’s greenhouse gas emissions have increased by 48% since 2019.

Energy companies are raising electricity prices to fund building more energy infrastructure to support these data centers. Furthermore, these data centers have placed an incredible strain on the grid, increasing the chances of electricity blackouts during peak times.

Researchers at the University of California, Riverside, estimated last year that global AI demand could suck up 1.1 trillion to 1.7 trillion gallons of freshwater by 2027.

In one instance, in Oregon where Google runs three data centers and plans two more, Google filed a lawsuit, aided by the city government, to keep their water consumption a secret from farmers, environmentalists, and Native American tribes. After they faced pressure to release the data, they caved and the records were made public. They showed that Google’s three data centers use more than a quarter of the city’s water supply.

And “Lavender,” which is trained on data about known militants, then parses through surveillance data about almost everyone in Gaza — from photos to phone contacts — to rate each person’s likelihood of being a militant. While the IOF claims that this is still gated on a human making the final call, Israeli soldiers told +972 that they essentially treated the AI’s output “as if it were a human decision,” sometimes only devoting “20 seconds” to looking over a target before bombing, and that the army leadership encouraged them to automatically approve Lavender’s kill lists a couple weeks into the war.”

While the mainstream narrative around AI characterizes AI models as systems that can operate entirely autonomously and are freeing workers by building machines that can do the boring and repetitive tasks, this is actually quite far from the truth. Instead these AI companies are building their AI models by treating many workers like machines. Training these models requires enormous amounts of data, and much of that data is cleaned and annotated by humans. Tech companies leverage the economic disparities between regions and this work is often outsourced to workers in the Global South including Syria, Argentina and Kenya, where workers are paid less than USD 1.50 per hour, with little job security and no clear path to upward mobility, and no protections for workers rights.

Some of the types of tasks might involve watching content and assessing whether it needs to be flagged. This means the workers must watch video sometimes containing suicide, murder, child abuse, and sexual assault and many workers report having developing stress and anxiety disorders from being constantly exposed to this content.

A report from Morgan Stanley suggests the datacenter industry is on track to emit 2.5 billion tons by 2030, which is three times higher than the predictions if generative AI had not come into play.

The difficulty in mitigating the environmental impact of data centers is that they can reduce energy consumption through water-cooling systems, but it takes an enormous amount of water to do so. With water becoming a more precious resource, those systems hamper tech giant’s green goals and place huge strains on areas with ‘high water scarcity’.

Earlier this week, Google revealed that its emissions have increased by 48% in just five years, despite its previous promises to hit net-zero emissions by 2030. Much of that increase is driven by its push to accelerate AI deployment. The picture is much the same at Microsoft.

With its investment in OpenAI, Microsoft is at the forefront of the generative AI push and it’s also leading the charge to expand data center capacity to enable it. The company now has more than 5 gigawatts (GW) of installed server capacity, which is more than Hong Kong or Portugal’s electricity consumption, and it set an internal goal of significantly accelerating new capacity to an additional 1GW in the first half of 2024, reaching an additional 1.5GW to be added in the first half of 2025 alone. It reportedly spent more than $50 billion on new data center capacity between July 2023 and June 2024.

Tech companies once pitched themselves as the purveyors of a more ethical form of capitalism. They wanted us to believe they would balance corporate profit with environmental sustainability, such that the digital future was marketed as inherently more sustainable than the analog past. It’s clearer than ever that was a lie.

In January, Sam Altman told a crowd at the World Economic Forum that we had to vastly increase energy production to power AI regardless of the consequences, and that we could simply geoengineer the planet to try to minimize the climate impacts.

In a new study, UW researchers found that ChatGPT consistently ranked resumes with disability-related honors and credentials — such as the “Tom Wilson Disability Leadership Award” — lower than the same resumes without those honors and credentials.

When asked to explain the rankings, the system spat out biased perceptions of disabled people. For instance, it claimed a resume with an autism leadership award had “less emphasis on leadership roles” — implying the stereotype that autistic people aren’t good leaders.

She and her colleagues calculated that the large language model BLOOM emitted greenhouse gases equivalent to 19 kilograms of CO2 per day of use, or the amount generated by driving 49 miles in an average gas-powered car. They also found that generating two images with AI could use as much energy as the average smartphone charge. Others have estimated in research posted on the preprint server arXiv.org that every 10 to 50 responses from ChatGPT running GPT-3 evaporate the equivalent of a bottle of water to cool the AI’s servers.

Data centers, including those that house AI servers, currently represent about 1.5 percent of global energy usage but are projected to double by 2026, at which point they may collectively use as much power as the country of Japan does today.

data centers and AI are likely to consume half of all the electricity available from renewable energy resources such as solar and wind farms

There is a new wind farm off the coast of Scotland that is supposed to be able to power 1.3 million homes, but Amazon has just announced it has spoken for more than half of that installation’s 880 MW output.

Data centers need power for two primary purposes, Wired says. The first is to power the chips that enable computers to run algorithms or power video games. The second is to cool the servers so they don’t overheat.

At Davos this year, OpenAI CEO Sam Altman also warned that the status quo was not going to be able to provide AI with the power it needed to advance. “There’s no way to get there without a breakthrough,” he said at a Bloomberg event.

Nvidia's Blackwell cluster, which will come with eight GPUs, pulls down 15kW of power. That's 15,000 watts of power. Divided by eight, that's 1,875 watts per GPU.

Worse still, Huang said that in the future, he expects to see millions of these kinds of AI processors in use at data centers around the world.  One million Blackwell GPUs would suck down an astonishing 1.875 gigawatts of power. For context, a typical nuclear power plant only produces 1 gigawatt of power.

In one segment of the keynote, Huang talked about the potential for Nvidia ACE to power 'digital humans' that companies can use to serve as customer service agents, be the face of an interior design project, and more.

Nvidia ACE may ultimately be the face of the service industry in a decade, and all those workers it will replace will have few options for employment elsewhere, since AI – powered by Nvidia hardware – will be taking over many other kinds of work once thought immune to this kind of disruption.

The problem with this kind of thinking, as it was with the first Industrial Revolution, is that this treats people as obstacles to be cleared away.

A universal basic income won't even paper over this disruption, much less solve it, especially because the ones who will have to pay for a universal basic income won't be the enormous pool of displaced workers, it would be those same tech titans who put all these people out of work in the first place. We can't even get the rich to pay the taxes they owe now. What makes anyone think they'll come in and take care of us in the end, no strings attached?

I think the worst part of Huang's keynote wasn't that none of this mattered, it's that I don't think anyone in Huang's position is really thinking about any of this at all. I hope they're not, which at least means it's possible they can be convinced to change course. The alternative is that they do not care, which is a far darker problem for the world.

US Big Tech companies are systemically abusing and exploiting African workers. In Kenya, these US companies are undermining the local labor laws, the country’s justice system and violating international labor standards. Our working conditions amount to modern day slavery.

We do this work at great cost to our health, our lives and our families. US tech giants export their toughest and most dangerous jobs overseas. The work is mentally and emotionally draining.

We label images and text to train generative AI tools like ChatGPT for OpenAI. Our work involves watching murder and beheadings, child abuse and rape, pornography and bestiality, often for more than 8 hours a day. Many of us do this work for less than $2 per hour.

We weren’t warned about the horrors of the work before we started.

On Wednesday, 97 African workers who do AI training work or online content moderation for companies like Meta and OpenAI published an open letter to President Biden, demanding that US tech companies stop “systemically abusing and exploiting African workers.”

A typical workday for African tech contractors, the letter says, involves “watching murder and beheadings, child abuse and rape, pornography and bestiality, often for more than 8 hours a day.” Pay is often less than $2 per hour, it says, and workers frequently end up with post-traumatic stress disorder, a well-documented issue among content moderators around the world.

In its latest environmental report, Microsoft disclosed that its global water consumption spiked 34% from 2021 to 2022 (to nearly 1.7 billion gallons, or more than 2,500 Olympic-sized swimming pools), a sharp increase compared to previous years that outside researchers tie to its AI research.

In a paper due to be published later this year, Ren’s team estimates ChatGPT gulps up 500 milliliters of water (close to what’s in a 16-ounce water bottle) every time you ask it a series of between 5 to 50 prompts or questions.

Google reported a 20% growth in water use in the same period, which Ren also largely attributes to its AI work. Google’s spike wasn’t uniform -- it was steady in Oregon where its water use has attracted public attention, while doubling outside Las Vegas. It was also thirsty in Iowa, drawing more potable water to its Council Bluffs data centers than anywhere else.

In July 2022, the month before OpenAI says it completed its training of GPT-4, Microsoft pumped in about 11.5 million gallons of water to its cluster of Iowa data centers, according to the West Des Moines Water Works.

Popular narratives focus, not on what big-crunch machine learning models can do now, but what they theoretically could do in the future.

Reading, understanding, and fixing code written by others consumes 90+% of the time a programmer spends in an integrated development environment, command line, or observability interface.

If we use LLMs all the time, the amount of “fix someone else’s code” we’re doing goes from 90% of the time to 100% of the time.

Large language model purveyors and enthusiasts purport to use the tools to help understand code. I’ve tested this claim pretty thoroughly at this point, and my conclusion on the matter is this: much like perusing answers on StackOverflow, this approach saves you time relative to whether you’re already skilled enough to know when to be suspicious, because a large proportion of the answers won’t help you.

But like an IDE, or a framework, or a test harness, utility here requires skill on the part of the operator—and not just ChatGPT jockeying skill: programming skill. Existing subject matter expertise.

That’s not entirely the fault of the model design. It has a lot to do with the fact that the training data, the Common Crawl, feeds it the code debugging and demystification advice of The Internet. How good is the advice about this topic on The Internet? Not that good, because by and large, developers suck at this skill.

Models trained on human data can’t outperform the base error rate in that data.

Haas estimates that while US power consumption by AI datacenters sits at a modest four percent, he expects the industry to trend towards 20 to 25 percent usage of the US power grid by 2030, per a report from the Wall Street Journal. He specifically lays blame at popular large language models (LLMs) such as ChatGPT, which Haas described as "insatiable in terms of their thirst."

OpenAI's GPT-4 large language model (LLM) can autonomously exploit vulnerabilities in real-world systems if given a CVE advisory describing the flaw.

Kang said he expects LLM agents, created by (in this instance) wiring a chatbot model to the ReAct automation framework implemented in LangChain, will make exploitation much easier for everyone. These agents can, we're told, follow links in CVE descriptions for more information.

Strikingly, the only text the manager actually inputs herself is the headline — "Best Bicycles for Kids" — and a series of links to Amazon products.Then the AI generates every single word of the article — "riding a bike is a right of passage that every child should experience," MEL advises, adding that biking teaches children important "life skills" like "how to balance and how to be responsible for their actions" — as the manager clicks buttons like "Generate Intro Paragraph" and "Generate Product Awards."

By the end of the video, the manager has produced an article identical in structure to the AdVon content we found at Sports Illustrated and other AdVon-affiliated publications: an intro, followed by a string of generically-described products with affiliate links to Amazon, a "buying guide" packed with SEO keywords, and finally an FAQ.

All five of the microwave reviews include an FAQ entry saying it's okay to put aluminum foil in your prospective new purchase.

do its human employees actually try the products being recommended?"No," laughed one AdVon source. "No. One hundred percent no.""I didn't touch a single one," another recalled.

People searching Google to buy a product, Weissman explains, are easily swayed by reviews in authoritative publications.

CNET got lambasted for publishing dozens of AI-generated articles about personal finance before discovering they were riddled with errors and plagiarism.

UnitedHealthcare, the largest health insurance company in the US, is allegedly using a deeply flawed AI algorithm to override doctors' judgments and wrongfully deny critical health coverage to elderly patients.

But, instead of changing course, over the last two years, NaviHealth employees have been told to hew closer and closer to the algorithm's predictions. In 2022, case managers were told to keep patients' stays in nursing homes to within 3 percent of the days projected by the algorithm, according to documents obtained by Stat. In 2023, the target was narrowed to 1 percent.

If a nursing home balked at discharging a patient with a feeding tube, case managers should point out that the tube needed to provide "26 percent of daily calorie requirements" to be considered as a skilled service under Medicare coverage rules.

It’s unclear how much UnitedHealth saves by using nH Predict, but Stat estimated it to be hundreds of millions of dollars annually. In 2022, UnitedHealth Group’s CEO made $20.9 million in total compensation. Four other top executives made between about $10 and $16 million each.

Kaiser Permanente, for its part, insists it’s simply leveraging “state-of-the-art tools and technologies that support our mission of providing high-quality, affordable health care to best meet our members’ and patients’ needs.”

Implementing this kind of transformative but error-prone tech in an industry where lives are on the line requires patience, intelligent planning, broad consultation with every level of employee, and competent regulatory guidance, none of which are American strong suits of late.

The club had “no idea” the software, set up with no manual intervention, was that unreliable, Rosin said.

Many of these software companies promoting AI say that one line of code will be enough to ensure compliance.

Blind users often say that overlays can make websites harder to navigate or that they interfere with assistive technologies.

As with any tech fad, AI is being applied carelessly, as if it carried no cost, as if a product got better by just featuring it.

lowering the bar of what consumers are willing to accept —passing the Turing test by making people dumber.

Things that felt like boilerplate or required subtle knowledge of tools that I don’t use frequently1. Regular expressions, crontab schedules, nginx rules, GitHub Actions workflows, Makefile targets, sed and awk incantations, CSS tricks.

Here is how it works. You ask you favorite LLM: "How to download a file in Node.js?" The LLM answers you the following code: import { Client } from 'http-request'; const client = new Client(); const res = client.get('https://example.com'); await fs.WriteFile('file.txt', res.body); You try it, it works, great! So you include it into your project and call it a day. AI for the win!!! Not so fast! Where does this http-request package come from? Who controls it? Why did the LLM "used" it in the code? You just have been pwned: http-request was a backdoored package distributed by a malicious actor.

Remember those news items that said these services were running at a loss? Well, those calculations were based on pure compute costs, which DON’T include capital expenses like chips or hardware and it didn’t include labour costs which are, ironically, quite high for generative models. All of which means the economics for “AI” is much much worse than most people think.

Nurses know that AI technology and algorithms are owned by corporations that are driven by profit — not a desire to improve patient care conditions or advance the nursing profession

This technology forces RNs to respond to excessive, if not faulty, alerts – which sometimes mistakenly flag that a patient’s safety is in jeopardy –  rather than using their knowledge and skills of observation to assess how to meet the needs of all patients. Conversely, patients who are at risk for deterioration are often missed by AI technology, which would have otherwise been caught by a thorough hands-on assessment from highly-trained medical professionals

AI technology also automatically completes note-taking that can miss important details and nuances about the patient.

Patient care requires — and will always require — nurses. Life-and-death decisions relating to patient acuity, treatment decisions, and staffing levels cannot be made without the assessment skills and critical thinking of registered nurses. For example, tell-tale signs of a patient’s condition, such as the smell of a patient’s breath and their skin tone, affect, or demeanor, are often not detected by AI and algorithms.

AI technologies enable mass surveillance of nurses and other health care workers at facilities, with disturbing opportunities for employers to violate individual privacy and union organizing rights.