Found 81 bookmarks
Custom sorting
llama.cpp guide: running gpt-oss with llama.cpp
llama.cpp guide: running gpt-oss with llama.cpp
Really useful official guide to running the OpenAI gpt-oss models using llama-server from llama.cpp - which provides an OpenAI-compatible localhost API and a neat web interface for interacting with the …
·simonwillison.net·
llama.cpp guide: running gpt-oss with llama.cpp
qwen-image-mps
qwen-image-mps
Ivan Fioravanti built this Python CLI script for running the Qwen/Qwen-Image image generation model on an Apple silicon Mac, optionally using the Qwen-Image-Lightning LoRA to dramatically speed up generation. Ivan …
·simonwillison.net·
qwen-image-mps
Trying out llama.cpp’s new vision support
Trying out llama.cpp’s new vision support
This llama.cpp server vision support via libmtmd pull request—via Hacker News—was merged earlier today. The PR finally adds full support for vision models to the excellent llama.cpp project. It’s documented …
·simonwillison.net·
Trying out llama.cpp’s new vision support
An Intro to RAG with sqlite-vec & llamafile!
An Intro to RAG with sqlite-vec & llamafile!
A brief introduction to using llamafile (a single-file tool for working with large language models) and sqlite-vec (A SQLite extension for vector search) to build a Retrival Augmentation Generation (RAG) application. This was a live online event hosted on Dec 17th 2024 in the Mozilla AI Discord, join us for the next event at at https://discord.gg/Ve7WeCJFXk LINKS: - Doc w/ links to all mentioned projects/blog posts: https://docs.google.com/document/d/17GYLzlGUyJF9EDeaa1P-dFFZnkwxATnBcg5KnNgpvPE/edit?usp=sharing - Slides: https://docs.google.com/presentation/d/14Szda-VnZzepL-1U9Nb7sXQg_TTf56OQ-KtUIMQ5xug/edit?usp=sharing
·youtube.com·
An Intro to RAG with sqlite-vec & llamafile!