ai

Highly cited and useful papers related to machine learning, deep learning, AI, game theory, reinforcement learning - tirthajyoti/Papers-Literature-ML-DL-RL-AI

"Let me get this straight," he said. "You're hosting a 'future of AI' event in a city that has failed humanity so miserably?"

Silicon Valley is helping to accelerate the climate crisis in at least 3 major ways

Eryk Salvaggio says we must dispense with myths if we are to think more clearly about what AI actually is and does.

Demystifying attention, the key mechanism inside transformers and LLMs. Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support Special thanks to these supporters: https://www.3blue1brown.com/lessons/attention#thanks An equally valuable form of support is to simply share the videos. Demystifying self-attention, multiple heads, and cross-attention. Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support The first pass for the translated subtitles here is machine-generated, and therefore notably imperfect. To contribute edits or fixes, visit https://translate.3blue1brown.com/ And yes, at 22:00 (and elsewhere), "breaks" is a typo. ------------------ Here are a few other relevant resources Build a GPT from scratch, by Andrej Karpathy https://youtu.be/kCc8FmEb1nY If you want a conceptual understanding of language models from the ground up, @vcubingx just started a short series of videos on the topic: https://youtu.be/1il-s4mgNdI?si=XaVxj6bsdy3VkgEX If you're interested in the herculean task of interpreting what these large networks might actually be doing, the Transformer Circuits posts by Anthropic are great. In particular, it was only after reading one of these that I started thinking of the combination of the value and output matrices as being a combined low-rank map from the embedding space to itself, which, at least in my mind, made things much clearer than other sources. https://transformer-circuits.pub/2021/framework/index.html Site with exercises related to ML programming and GPTs https://www.gptandchill.ai/codingproblems History of language models by Brit Cruise,  @ArtOfTheProblem  https://youtu.be/OFS90-FX6pg An early paper on how directions in embedding spaces have meaning: https://arxiv.org/pdf/1301.3781.pdf ------------------ Timestamps: 0:00 - Recap on embeddings 1:39 - Motivating examples 4:29 - The attention pattern 11:08 - Masking 12:42 - Context size 13:10 - Values 15:44 - Counting parameters 18:21 - Cross-attention 19:19 - Multiple heads 22:16 - The output matrix 23:19 - Going deeper 24:54 - Ending ------------------ These animations are largely made using a custom Python library, manim. See the FAQ comments here: https://3b1b.co/faq#manim https://github.com/3b1b/manim https://github.com/ManimCommunity/manim/ All code for specific videos is visible here: https://github.com/3b1b/videos/ The music is by Vincent Rubinetti. https://www.vincentrubinetti.com https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown https://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u ------------------ 3blue1brown is a channel about animating math, in all senses of the word animate. If you're reading the bottom of a video description, I'm guessing you're more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise following on whichever platform below you check most regularly. Mailing list: https://3blue1brown.substack.com Twitter: https://twitter.com/3blue1brown Instagram: https://www.instagram.com/3blue1brown Reddit: https://www.reddit.com/r/3blue1brown Facebook: https://www.facebook.com/3blue1brown Patreon: https://patreon.com/3blue1brown Website: https://www.3blue1brown.com

Did you know you can play Doom on a diffusion model now? It’s true, Google just announced it! Just don’t read the paper too closely. In their paper “Diffusion models are real-time game engines,” Go…

Negli ultimi decenni, il mondo della robotica ha visto enormi cambiamenti, portando alla nascita di nuove tecnologie e, soprattutto, di nuove sfide etiche e legali. Isaac Asimov, celebre scrittore di fantascienza, aveva anticipato queste problematiche negli anni ’40, quando formulò le sue tre leggi della robotica. Queste leggi immaginate da Asimov, nonostante siano state sviluppate …

Nature - Despite efforts to remove overt racial prejudice, language models using artificial intelligence still show covert racism against speakers of African American English that is triggered by...

Especially in IT and software development numbers keep popping up about “savings” through AI. Amazon for example claims to have “saved” 4500 person years of work. These numbers have to be taken with a grain of salt and shouldn’t be interpreted as “oh, we will save massive amounts of work by using AI, let’s fire […]

History of AI Symbolism vs Parallel Processing Nicholas K. Gracilla Denison University, 1993

A schism lies at the heart of the field of artificial intelligence. Since its inception, the field has been defined by an intellectual tug-of-war between two opposing philosophies: connectionism and symbolism. These two camps have deeply divergent visions as to how to "solve" intelligence, with differing research agendas and sometimes bitter relations.

Copilot soll als Alltagshelfer, kreativer Assistent und für den Einsatz im Büro dienen, so die Werbung. In den AGB klingt das allerdings anders.

“But if nobody uses your product, it doesn’t matter that you stole all the content.”

“Language models are just glorified autocomplete” has been the critics’ refrain — but reinforcement learning is proving them wrong. New breakthroughs could follow.

Forscher fanden heraus, dass sich KI-Modelle selbst sabotieren, indem sie KI-generierte Daten zum Training verwenden. Sie produzieren dann immer mehr Müll.

Trotz der hohen Erwartungen führen verschiedene Hürden dazu, dass Unternehmen GenAI-Initiativen bereits in der Pilotphase abbrechen.

This page contains an annotated list of my publications/explanations about the 2023 surge of attention (hype is not always a wrong word here) society paid to Artificial Intelligence, mostly because…

One of the use cases I thought was reasonable to expect from ChatGPT and Friends (LLMs) was summarising. It turns out I was wrong. What ChatGPT isn’t summarising at all, it only looks like it…

The growing carbon footprint of artificial intelligence (AI) models, especially large ones such as GPT-3, has been undergoing public scrutiny. Unfortunately, however, the equally important and enormous water (withdrawal and consumption) footprint of AI models has remained under the radar. For example, training GPT-3 in Microsoft's state-of-the-art U.S. data centers can directly evaporate 700,000 liters of clean freshwater, but such information has been kept a secret. More critically, the global AI demand may be accountable for 4.2 -- 6.6 billion cubic meters of water withdrawal in 2027, which is more than the total annual water withdrawal of 4 -- 6 Denmark or half of the United Kingdom. This is very concerning, as freshwater scarcity has become one of the most pressing challenges shared by all of us in the wake of the rapidly growing population, depleting water resources, and aging water infrastructures. To respond to the global water challenges, AI models can, and also must, take social responsibility and lead by example by addressing their own water footprint. In this paper, we provide a principled methodology to estimate the water footprint of AI models, and also discuss the unique spatial-temporal diversities of AI models' runtime water efficiency. Finally, we highlight the necessity of holistically addressing water footprint along with carbon footprint to enable truly sustainable AI.

It’s important that we use accurate terminology when discussing how AI chatbots make up information

500.000 Tonnen CO2 sollen im Auftrag Microsofts in Texas aus der Luft gesaugt und in den Boden gepumpt werden. Ein Rekord.​

Researchers have been probing water consumption figures associated with Microsoft's data center in Iowa.

OpenAI is begging Parliament to allow it to use copyrighted works because it's "impossible" for the company to make money without them.

AI companies keep claiming that LLMs do reasoning. The AI can think! The AI can figure out word puzzles! The AI can pass the LSAT! You’re fired! So our good friend Diz tested this by throwing logic…

Ohne Millionen Datenarbeiter:innen würden weder sogenannte Künstliche Intelligenz noch Content-Moderation funktionieren. In einem neuen Projekt erzählen sie ihre Geschichten: von Plattformarbeiter:innen in Venezuela und Syrien über Angestellte von Outsourcing-Firmen in Kenia bis zu Content-Moderator:innen in Deutschland.

Wichtiger Beitrag über die digitale Drecksarbeit von Content-Moderator*innen. https://www.spiegel.de/netzwelt/web/digitale-drecksarbeit-die-belastende-arbeit-der-content-moderatoren-in-afrika-a-7dd3f260-8bb3-40e3-95a3-adf4137eaad3

Google researchers have come out with a new paper that warns that generative AI is ruining vast swaths of the internet with fake content.

This is the first time the carbon emissions caused by using an AI model for different tasks have been calculated.