AIxDESIGN Bookmark Library

An important paradigm of natural language processing consists of large-scale pre-training on general domain data and adaptation to particular tasks or domains. As we pre-train larger models, full fine-tuning, which retrains all model parameters, becomes less feasible. Using GPT-3 175B as an example -- deploying independent instances of fine-tuned models, each with 175B parameters, is prohibitively expensive. We propose Low-Rank Adaptation, or LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks. Compared to GPT-3 175B fine-tuned with Adam, LoRA can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times. LoRA performs on-par or better than fine-tuning in model quality on RoBERTa, DeBERTa, GPT-2, and GPT-3, despite having fewer trainable parameters, a higher training throughput, and, unlike adapters, no additional inference latency. We also provide an empirical investigation into rank-deficiency in language model adaptation, which sheds light on the efficacy of LoRA. We release a package that facilitates the integration of LoRA with PyTorch models and provide our implementations and model checkpoints for RoBERTa, DeBERTa, and GPT-2 at https://github.com/microsoft/LoRA.

The premise of the paper is that while OpenAI and Google continue to race to build the most powerful language models, their efforts are rapidly being eclipsed by the work happening in the open source community.

SPACE10 is a research and design lab on a mission to create a better everyday life for people and the planet.

This Master in Artificial Intelligence explores the links between artificial intelligence, sustainability, ethics and social justice.

A community blog devoted to refining the art of rationality

Perplexity AI unlocks the power of knowledge with information discovery and sharing.

synthographic motion picture generated with Stable Diffusion - 5.5.2023 - a story inspired by "I, We, Waluigi: a Post-Modern analysis of Waluigi" by Franck Ribery https://theemptypage.wordpress.com/2013/05/20/critical-perspectives-on-waluigi/ #roaroar

Map and directory of PIAI projects

Bringing sharper focus to AI modeling. A user-guided interface that focuses on the problem space rather than the tools.

Towards open, democratic and sustainable digital public spaces

*Public administrators, public tech developers, and public service providers face the same challenge: How to develop and use technology in accordance with public values like openness, fairness, and inclusivity? The question is urgent as we continue to rely upon proprietary technology that is developed within a surveillance capitalist context and is incompatible with the goals and missions of our democratic institutions. This problem has been a driving force behind the development of the [public stack](https://publicstack.net/), a conceptual model developed by [Waag](https://waag.org/en/) through [ACROSS ]()and other projects, which roots technical development in public values.* The idea behind the public stack is simple: There are unseen [layers](https://publicstack.net/layers/) behind the technology we use, including hardware, software, design processes, and business models. All of these layers affect the relationship between people and technology – as consumers, subjects, or (as the public stack model advocates) citizens and human beings in a democratic society. The public stack challenges developers, funders, and other stakeholders to develop technology based on shared public values by utilising participatory design processes and open technology. The goal is to position people and the planet as democratic agents; and as more equal stakeholders in deciding how technology is developed and implemented. ACROSS is a Horizon2020 European project that develops open source resources to protect digital identity and personal data across European borders. In this context, Waag is developing the public stack model into a service design approach – a resource to help others reflect upon and improve the extent to which their own ‘stack’ is reflective of public values. In late 2022, Waag developed a method using the public stack as a lens to prompt reflection amongst developers. A more extensive public stack reflection process is now underway in ACROSS; resources to guide other developers through this same process will be made available later in 2023. The public stack is a useful model for anyone involved in technology, whether as a developer, funder, active, or even passive user. In the case of ACROSS, its adoption helped project partners to implement decentralised privacy-by-design technology based on values like privacy and user control. The model lends itself to be applied just as well in other use cases: * Municipalities can use the public stack to maintain democratic approaches to technology development and adoption in cities. * Developers of both public and private tech can use the public stack to reflect on which values are embedded in their technology. * Researchers can use the public stack as a way to ethically assess technology. * Policymakers can use the public stack as a way to understand, communicate, and shape the context in which technology development and implementation occurs. ***Are you interested in using the public stack in your own project, initiative, or development process? We’d love to hear about it. Let us know more by emailing us at publicstack@waag.org.***

Erasmus Initiative Societal Impact of AI aims for innovative and interdisciplinary research and education in AI that places people and society centre stage.

internet teapot wants to re-think how we can make our digital present and the future can be reshaped to be more human, more conscientious, and more sustainable.

We want to make study and work more efficient and enjoyable, by providing the best digital paper solution possible. We plan to be the go-to tool for all forms of notes.

Browse words and phrases for each section in your paper or thesis: from Abstract to Acknowledgements.

“Can AI help us imagine a more desirable future? We’ve been exploring how generative AI tools can help us think differently about our future homes. All images were generated with the assistance of Midjourney. Prompts in the ALT text. #AIarchitecture #Midjourney”

Our challenge is to adapt Artificial Intelligence to new business and human needs with social and environmental responsibility.

The creative reach of the individual is expanding.

Special thanks to Kate Woolverton for comments and feedback. …

Hi everyone! • I'm co-organizing a workshop on a really interesting topic that's very relevant for AI safety. We call it "Rebellion and Disobedience in AI".  …

Collected data and analysis from a large survey of machine learning researchers.

AVERTING DOOM BY NOT BUILDING THE DOOM MACHINE If you fear that someone will build a machine that will seize control of the world and annihilate humanity, then one kind of response is to try to build…

Fontjoy helps designers choose the best font combinations. Mix and match different fonts for the perfect pairing.

Artist and Writer | Portfolio

The machines that nudge us can be nudged back: Here are some ideas on how to steer them.