System Architecture

I’m in the middle of a rather large project, attempting to consolidate a thread running through roughly a dozen essays I’ve written over the past decade or so. I keep running into borke…

With e-commerce at the forefront nowadays, the need for innovative payment solutions is higher than ever. One such challenge is a dispersed…

The Frugal Architect returns with expanded content, featuring stories from AWS customers optimizing their architectures for cost and sustainability.

Master the internals of Apache ZooKeeper through comprehensive source code analysis. Learn about distributed coordination, consensus algorithms, and real-world implementation patterns.

Build iOS, Android, and Progressive Web Apps with HTML, CSS, and JavaScript

Exploring the benefits of using archives (ZIP/RAR) in Single Page Applications (SPA) to simplify resource management, communication between client and server, and enhance the separation of front-end and back-end.

This paper describes the GNU make “jobserver” implementation: it is meant mainly for people who want to understand the GNU make jobserver, but also for those interesting in a traditiona…

Agoda recently described their innovative approach to transitioning from a monolithic GraphQL API to a microservices architecture. Unlike traditional methods focusing on breaking down server-side components first, Agoda adopted a client-first strategy, preparing their client applications to handle both the monolith and the microservices in parallel using an in-house smart orchestrator library.

Except when they are not. I'll try explain some the underlying concepts, so you can use them better, and sound really cool with your friends and coworkers.

Despite all its strengths, SQL can be awkward to integrate with host languages such as JavaScript and Python. There’s often an impedance mismatch between SQL’s declarative nature and the host language’s object-oriented or functional paradigms – SQL queries are typically written as strings within the host language, making it difficult to apply composable programming techniques…

The Model Context Protocol (MCP) is an open standard for connecting AI assistants to the systems where data lives, including content repositories, business tools, and development environments. Its aim is to help frontier models produce better, more relevant responses.

Authentication is the foundation of all web services and must be a top priority in terms of security, feature completeness, and UX.

Vector databases are quite popular right now, especially for building recommendation systems, adding context to chatbots and LLMs, or comparing content based on similarity. In this guide, I'll explain what vector databases are, how they work, and when to use them.

I was frustrated by the gap between stateless and stateful applications in the cloud. While I could easily spin up a stateless application as a “serverless” function in any major cloud provider and pretty much forget about it, persisting data between requests was a game of pick two among three: cheap, strongly consistent, portable. Could I solve portability and lack of transactions myself with a single client-side solution? I thought it would be possible through object storage (e.g. AWS S3), which is strongly consistent, ubiquitous and cheap.

Explore how service discovery and load balancing play crucial roles in microservices for dynamic communication and connection between multiple services.

This article provides ten essential rules to optimize your CI pipeline for efficiency, reliability, and code quality.

To realize the Internet of Everything (IoE) revolution, enterprises must be able to deploy item-level intelligence at scale. Flexible chips are the key.

One tier worse than regular properties.

When we design programs, we usually look for two kinds of properties: that “bad things” never happen and that “good things” are guaranteed to happen. These are called safety and liveness properties, respectively. These are properties that we want to hold true for every possible program behavior. “We always complete every request” is a liveness property. If our system has it, every program trace will complete every request. If it doesn’t hold, I can give you a example behavior where the server never responds.

I’ve noticed some confusion about using SPM to model graphs with local packages, so I wanted to shed some light on the topic. While SPM can describe a project graph, we believe it’s not the best tool for the job—at least not today. SPM’s design decisions make sense in the context of package resolution, but they fall short when it comes to project management. If you want full control over your graph and the developer experience within your project, we recommend limiting SPM to its core purpose: ...

State is pain. The next generation of infrastructure tools will be built on diskless paradigm. In this short post I will explain what is Diskless / Zero Disk Architecture

A functional Scheme for world domination

PRQL is a modern language for transforming data

Let’s say you created a Go program that stores data in PostgreSQL — you installed PostgreSQL, wrote the Go code, and everything works; great! But after writing a test for your code, you wonder: how do you best provide PostgreSQL to your automated tests?

Using a full-featured RPC framework for IPC seems like overkill when the processes run on the same machine. However, if your project anyway exposes RPCs for public APIs or would benefit from a schema-based serialisation layer it makes sense to use only one tool that combines these—also for IPC. Microservices for beginners For the FlashCam software running on the DAQ servers we converged on the following high-level architecture: Each bright box corresponds to a software process running on the DAQ server.

Understanding Database Failure Handling Using Write-Ahead Logs