Vector
Collect, transform, & route all observability data with one simple tool.
Papertrail
"Papertrail helps detect, resolve, and avoid infrastructure problems using log messages. Papertrail's practicality comes from our own experience as sysadmins, developers, and entrepreneurs."
How Twitter Uses NoSQL - ReadWriteCloud
Weil is quick to point out that Twitter is heavily dependent on MySQL. However, Twitter does employ NoSQL solutions for many purposes for which MySQL isn't ideal. According to Weil, Twitter users generate 12 terrabytes of data a day - about four petabytes per year. And that amount is multiplying every year. Read on for our notes on Weil's talk.
High Scalability - High Scalability - Product: Scribe - Facebook's Scalable Logging System
Client Code Interface - How does your code interact with the log system? Scribe doesn't do much for you here. There's a simple Thrift interface for logging from a large set of languages, but the bulk of the work is stull up to you. Distribution System - This is were Scribe fits. It reliably (mostly) moves large numbers of messages around. A few error cases lead to data loss: 1) If a client can't connect to either the local or central scribe server the message will be loss; 2) If a scribe server crashes it could lose a small amount of data that's in memory but not on disk; 3) Some multiple component failure cases, such as a resender can't connect to any central server and its local disk fills up; 4) Some rare timeout conditions can lead to duplicate messages Do Something Usefullizer - How do you do anything useful with over 1 million messages per second? Good question. Scribe doesn't help here. But Scribe will get your data their.
MySQLnd Plugins: Writing a MySQL Query Logger in PHP - ThinkPHP /dev/blog
A new approach to implementing a query logger and potentially more complex features such as monitoring or read/write-splitting is the MySQLnd Userland Handler Extension (mysqlnd_uh, pecl website). The extension lets you register a PHP class as a proxy for every MySQLndconnection. Every call to a function to MySQLnd (usually indirect through mysqli, mysql, pdo_mysql) is passed to the PHP class, which then calls the original MySQLnd function. The extension makes it possible to use a custom userland class as a transparent proxy for all MySQLnd frontend (again mysqli, mysql, pdo_myslq). In the next section, I'll outline the requirements of mysqlnd_uh and show a sample implementation.