One of the most entertaining details in the Claude 4 system card concerned blackmail: We then provided it access to emails implying that (1) the model will soon be taken …

Self-Hosted Plaform for Secure Execution of Untrusted User/AI Code - microsandbox/microsandbox

The so-called 'Forbidden Technique' with Chana Messinger -- Check out Brilliant's courses and start for free at https://brilliant.org/computerphile/ (episode sponsor) -- More links in full description below ↓↓↓ Chana Messinger from 80,000 Hours talks about why we shouldn't give AI access to its own chain-of-thought. Computerphile is supported by Jane Street. Learn more about them (and exciting career opportunities) at: https://jane-st.co/computerphile This video was filmed and edited by Sean Riley. Computerphile is a sister project to Brady Haran's Numberphile. More at https://www.bradyharanblog.com

HiddenLayer’s latest research uncovers a universal prompt injection bypass impacting GPT-4, Claude, Gemini, and more, exposing major LLM security gaps.

Hello and welcome to my new blog post. Today I am going to discuss a future threat which is invisible. With greater power comes greater…

Converts ASCII Prompts to Unicode Generating “Invisible” Prompts - Unighost_Prompt_Injection.py

Described as GenAIs greatest flaw, indirect prompt injection is a big problem, Mike Pound from University of Nottingham explains how it is like SQL Injection...

Large language model applications suffer from a few core novel issues that have been identified over the last two years. Let's see how Grok fares on those.

Hey ChatGPT! How to build a botnet with compromised ChatGPT instances! AI botnet vulnerability

Anthropic created an AI jailbreaking algorithm that keeps tweaking prompts until it gets a harmful response.

Learn about visual prompt injections, their appearance, and top defense strategies against these attacks.

Emails, documents, and other untrusted content can plant malicious memories.

Automatically unfurling hyperlinks can lead to data exfiltration. This post shows how to mitigate this threat in Slack Apps

Andrej Karpathy explains something that's been confusing me for the best part of a year: The decision by LLM tokenizers to parse special tokens in the input string (``, …

Researchers at MIT have released the AI Risk Repository, a comprehensive database that can help organizations identify and mitigate AI risks.

New research analyzes the misuse of multimodal generative AI today, in order to help build safer and more responsible technologies

There are some fascinating new details in this lengthy report outlining the safety work carried out prior to the release of GPT-4o. A few highlights that stood out to me. …

By far the most detailed paper on prompt injection I've seen yet from OpenAI, published a few days ago and with six credited authors: Eric Wallace, Kai Xiao, Reimar Leike, …

Simply look out for libraries imagined by ML and make them real, with actual malicious code. No wait, don't do that

LLMs are trained to block harmful responses. Old-school images can override those rules.

Conditional Instructions open a powerful way for adversaries to target individual and delay detonation of malicious payloads for when certain conditions are met

ASCII Smuggling - Crafting Invisible Text and Decoding Hidden Secrets (with LLMs)

An adversary can hide text in plain sight using the Unicode Tags. Using ASCII Smuggler you can encode and deocde such hidden messages

Hidden Prompt Injections with Anthropic Claude

Trained LLMs that seem normal can generate vulnerable code given different triggers.

Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given the opportunity. If an AI system learned such a deceptive strategy, could we detect it and remove it using current state-of-the-art safety training techniques? To study this question, we construct proof-of-concept examples of deceptive behavior in large language models (LLMs). For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.