Bookmarks

Bookmarks

46714 bookmarks
Custom sorting
Defeating Nondeterminism in LLM Inference
Defeating Nondeterminism in LLM Inference
Reproducibility is a bedrock of scientific progress. However, it’s remarkably difficult to get reproducible results out of large language models. For example, you might observe that asking ChatGPT the same question multiple times provides different results. This by itself is not surprising, since getting a result from a language model involves “sampling”, a process that converts the language model’s output into a probability distribution and probabilistically selects a token. What might be more surprising is that even when we adjust the temperature down to 0This means that the LLM always chooses the highest probability token, which is called greedy sampling. (thus making the sampling theoretically deterministic), LLM APIs are still not deterministic in practice (see past discussions here, here, or here). Even when running inference on your own hardware with an OSS inference library like vLLM or SGLang, sampling still isn’t deterministic (see here or here).
·thinkingmachines.ai·
Defeating Nondeterminism in LLM Inference
Claude Memory: A Different Philosophy
Claude Memory: A Different Philosophy
A breakdown of Claude's memory implementation, how it differs from ChatGPT, and what these opposite approaches reveal about each company's philosophy.
·shloked.com·
Claude Memory: A Different Philosophy
生成AI時代のサービス運営管理 - MCP Server for Administratorの実践 - - Findy Tech Blog
生成AI時代のサービス運営管理 - MCP Server for Administratorの実践 - - Findy Tech Blog
こんにちは。 ファインディ株式会社 で Tech Lead をやらせてもらってる戸田です。 現在のソフトウェア開発の世界は、生成AIの登場により大きな転換点を迎えています。 GitHub CopilotやClaude Codeなど生成AIを活用した開発支援ツールが次々と登場し、開発者の日常的なワークフローに組み込まれつ…
·tech.findy.co.jp·
生成AI時代のサービス運営管理 - MCP Server for Administratorの実践 - - Findy Tech Blog
A postmortem of three recent issues \ Anthropic
A postmortem of three recent issues \ Anthropic
This is a technical report on three bugs that intermittently degraded responses from Claude. Below we explain what happened, why it took time to fix, and what we're changing.
·anthropic.com·
A postmortem of three recent issues \ Anthropic
A deep dive into Cloudflare’s September 12, 2025 dashboard and API outage
A deep dive into Cloudflare’s September 12, 2025 dashboard and API outage
Cloudflare’s Dashboard and a set of related APIs were unavailable or partially available for an hour starting on Sep 12, 17:57 UTC. The outage did not affect the serving of cached files via the Cloudflare CDN or other security features at the Cloudflare Edge.
·blog.cloudflare.com·
A deep dive into Cloudflare’s September 12, 2025 dashboard and API outage
You don’t need quantum hardware for post-quantum security
You don’t need quantum hardware for post-quantum security
Post-quantum cryptography protects against quantum threats using today’s hardware. Quantum tech like QKD may sound appealing, but it isn’t necessary or sufficient to secure organizations.
·blog.cloudflare.com·
You don’t need quantum hardware for post-quantum security
Best practices for securing MCP model-agent interactions — WorkOS
Best practices for securing MCP model-agent interactions — WorkOS
A practical guide to securing MCP model–agent interactions: prevent prompt injection, privilege escalation, replay attacks, and data exfiltration with validation gateways, signing, DLP, and scoped creds.
·workos.com·
Best practices for securing MCP model-agent interactions — WorkOS
How we made global routing faster with Bloom filters - Vercel
How we made global routing faster with Bloom filters - Vercel
We replaced slow JSON path lookups with Bloom filters in our global routing service, cutting memory usage by 15% and reducing 99th percentile lookup times from hundreds of milliseconds to under 1 ms. Here’s how we did it.
·vercel.com·
How we made global routing faster with Bloom filters - Vercel
knadh/koanf: Simple, extremely lightweight, extensible, configuration management library for Go. Supports JSON, TOML, YAML, env, command line, file, S3 etc. Alternative to viper.
knadh/koanf: Simple, extremely lightweight, extensible, configuration management library for Go. Supports JSON, TOML, YAML, env, command line, file, S3 etc. Alternative to viper.
Simple, extremely lightweight, extensible, configuration management library for Go. Supports JSON, TOML, YAML, env, command line, file, S3 etc. Alternative to viper. - knadh/koanf
·github.com·
knadh/koanf: Simple, extremely lightweight, extensible, configuration management library for Go. Supports JSON, TOML, YAML, env, command line, file, S3 etc. Alternative to viper.
Building a Simple Virtual Machine | Breakpoint 🛑
Building a Simple Virtual Machine | Breakpoint 🛑
Explore building a simple VM in Go using stack-based architecture. Learn the fundamentals of virtual machines through hands-on implementation.
·blog.phakorn.com·
Building a Simple Virtual Machine | Breakpoint 🛑