AI/ML

AI/ML

35 bookmarks
Custom sorting
How To Scale Your Model
How To Scale Your Model
Training LLMs often feels like alchemy, but understanding and optimizing the performance of your models doesn't have to. This book aims to demystify the science of scaling language models on TPUs: how TPUs work and how they communicate with each other, how LLMs run on real hardware, and how to parallelize your models during training and inference so they run efficiently at massive scale. If you've ever wondered “how expensive should this LLM be to train or “how much memory do I need to serve this model myself” or “what's an AllGather”, we hope this will be useful to you.
·jax-ml.github.io·
How To Scale Your Model
Terence Tao, mathematician: ‘It’s not good for something as important as AI to be a monopoly held by one or two companies’ | Science | EL PAÍS English
Terence Tao, mathematician: ‘It’s not good for something as important as AI to be a monopoly held by one or two companies’ | Science | EL PAÍS English
The Fields Medal winner is attempting to solve one of the Millennium Problems, with a reward of $1 million, but he also applies his analysis to topical enigmas such as the Venezuelan election and the advance of artificial intelligence
·english.elpais.com·
Terence Tao, mathematician: ‘It’s not good for something as important as AI to be a monopoly held by one or two companies’ | Science | EL PAÍS English
The Chinese Room - 60-Second Adventures in Thought (3/6)
The Chinese Room - 60-Second Adventures in Thought (3/6)
An argument against computers ever being truly intelligent. (Part 3 of 6) Playlist link - https://www.youtube.com/playlist?list=PL73A886F2DD959FF1 Transcript link - http://podcast.open.ac.uk/feeds/thoughtexperiments-01/transcript/60second03_01691_16695.pdf Study a free course on Introducing philosophy at the Open University https://www.open.edu/openlearn/history-the-arts/culture/philosophy/introducing-philosophy/content-section-0?active-tab=description-tab Study R14 BA (Honours) Arts and Humanities (Philosophy) http://www.open.ac.uk/courses/qualifications/r14-p Explore qualifications in Philosophy with the OU http://www.open.ac.uk/courses/find/philosophy The Open University is the world’s leading provider of flexible, high-quality online degrees and distance learning, serving students across the globe with highly respected degree qualifications, and the triple-accredited MBA. The OU teaches through its own unique method of distance learning, called ‘supported open learning’ and you do not need any formal qualifications to study with us, just commitment and a desire to find out what you are capable of. Free learning from The Open University http://www.open.edu/openlearn/ For more like this subscribe to the Open University channel https://www.youtube.com/channel/UCXsH4hSV_kEdAOsupMMm4Qw Like us on Facebook: https://www.facebook.com/ouopenlearn/ Follow us on Twitter: https://twitter.com/OUFreeLearning #OpenUniversity #paradox
·youtube.com·
The Chinese Room - 60-Second Adventures in Thought (3/6)
Thread by @altryne on Thread Reader App
Thread by @altryne on Thread Reader App
@altryne: Watching @karpathy presentation from today and taking twitter notes, come along for the ride: If you're like only the practical tips, skip to #32 @karpathy starts with stages: 1 - Pre-training - months x th...…
·threadreaderapp.com·
Thread by @altryne on Thread Reader App
Understanding GPT tokenizers
Understanding GPT tokenizers
Large language models such as GPT-3/4, LLaMA and PaLM work in terms of tokens. They take text, convert it into tokens (integers), then predict which tokens should come next. Playing …
·simonwillison.net·
Understanding GPT tokenizers
Prompt Engineering
Prompt Engineering
Prompt Engineering, also known as In-Context Prompting, refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights. It is an empirical science and the effect of prompt engineering methods can vary a lot among models, thus requiring heavy experimentation and heuristics. This post only focuses on prompt engineering for autoregressive language models, so nothing with Cloze tests, image generation or multimodality models.
·lilianweng.github.io·
Prompt Engineering
Advanced AI Guide by The Rundown.
Advanced AI Guide by The Rundown.
Thanks for signing up to The Rundown built by @therundownai and @rowancheung! Enjoy our free Advanced ChatGPT Guide as a warm welcome into the world of AI!
·vaulted-polonium-23c.notion.site·
Advanced AI Guide by The Rundown.