virtulis / Prey / overview — bitbucket.org
PHP Redis Ymplementation. Yes, one more. This time with UDP and asynchronous stuff.
Tools
MyTAP: Unit Testing for MySQL
MyTAP is a unit testing framework for MySQL 5.x written using fuctions and procedures. It includes a collection of TAP-emitting assertion functions, as well as the ability to integrate with other TAP-emitting test frameworks.
A/B testing with PHP - phpA/B
Super simple setup Setting up an A/B test couldn’t be easier. Almost everything is done for you. You can be running multiple tests in only a matter of minutes.
Test anything Whether you want to test changes to your copy, HTML, CSS, javascript, or test completely different designs, phpA/B offers a way to do it.
Google Analytics integration phpA/B integrates automatically into your Google Analytics, allowing deeper insight into your tests’ impact on users.
High performance PHP-based tests run before users even begin loading your web pages. Javascript-based A/B tests cause funny twitches in your UI.
Search bot filtering Don’t let search engine robots skew your data. phpA/B automatically filters them out of your tests.
(Infogami)
Infogami is a wiki application framework built on web.py.
Front Page at innotop
innotop can monitor InnoDB transactions and internals, queries and processes, deadlocks, foreign key errors, replication status, system variables and status and much more.
VCSWrapper - Arbit - next generation project tracking
The VCS implements an abstraction layer for read access to various version control systems. It is meant to offer generic support for as many VCS as possible.
Krati
Krati is a simple persistent content data store with very low latency and high throughput. It is designed for easy integration with other services with little effort in tuning configuration, performance and JVM garbage collection. This software is published under the terms of the Apache Software License version 2.0, a copy of which has been included in the LICENSE file shipped with the Krati distribution.
orieg's php-judy at master - GitHub
php-judy is an exentsion by Nicolas Brousse for the Judy C library.
http://github.com/orieg/php-judyA Judy array is a complex but very fast associative array data structure for storing and looking up values using integer or string keys. Unlike normal arrays, Judy arrays may be sparse; that is, they may have large ranges of unassigned indices.
http://en.wikipedia.org/wiki/Judy_arrayJudy is a C library that implements a dynamic array. Empty Judy arrays are declared with null pointers. A Judy array consumes memory only when populated yet can grow to take advantage of all available memory. Judy's key benefits are: scalability, performance, memory efficiency, and ease of use. Judy arrays are designed to grow without tuning into the peta-element range, scaling near O(log-base-256) -- 1 more RAM access at 256 X population.
http://judy.sourceforge.netrelevance's cap_gun at master - GitHub
Tell everyone about your releases! Send email notification after Capistrano deployments! Rule the world!
Drop your ActionMailer configuration information into your deploy.rb file, configure recipients for the deployment notifications, and setup the callback task.
Setup and configuration are done entirely inside your deploy.rb file to keep it super simple. Your emails are sent locally from the box performing the deployment, but CapGun queries the server to grab the necessary release info.
This even includes the Net::SMTP TLS hack inside as a vendored dependancy to allow super easy email sending without setting up an MTA.
MySQLnd Plugins: Writing a MySQL Query Logger in PHP - ThinkPHP /dev/blog
A new approach to implementing a query logger and potentially more complex features such as monitoring or read/write-splitting is the MySQLnd Userland Handler Extension (mysqlnd_uh, pecl website). The extension lets you register a PHP class as a proxy for every MySQLndconnection. Every call to a function to MySQLnd (usually indirect through mysqli, mysql, pdo_mysql) is passed to the PHP class, which then calls the original MySQLnd function. The extension makes it possible to use a custom userland class as a transparent proxy for all MySQLnd frontend (again mysqli, mysql, pdo_myslq). In the next section, I'll outline the requirements of mysqlnd_uh and show a sample implementation.
High Scalability - High Scalability - Scaling an AWS infrastructure - Tools and Patterns
How do you scale an AWS (Amazon Web Services) infrastructure? This article will give you a detailed reply in two parts: the tools you can use to make the most of Amazon’s dynamic approach, and the architectural model you should adopt for a scalable infrastructure.
Test your Facebook application | FriendRunner
One of the great difficulties of Facebook application development is the difficulty in running large-scale tests due to the large number of Facebook accounts required, and the Facebook Terms of Use which does not allow automated systems to access the Facebook servers. These issues are explained in depth in Load testing Facebook apps (Part 1) and Load testing Facebook apps (Part 2). FriendRunner acts as a drop-in replacement for the Facebook servers, and so allows you to run your tests with as many users as you'd like. Furthermore, FriendRunner's automated test system runs the entire test for you once you create a test script to define the actions for a single user.
Welcome to Pig!
Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets.
At the present time, Pig's infrastructure layer consists of a compiler that produces sequences of Map-Reduce programs, for which large-scale parallel implementations already exist (e.g., the Hadoop subproject). Pig's language layer currently consists of a textual language called Pig Latin, which has the following key properties:
Ease of programming. It is trivial to achieve parallel execution of simple, "embarrassingly parallel" data analysis tasks. Complex tasks comprised of multiple interrelated data transformations are explicitly encoded as data flow sequences, making them easy to write, understand, and maintain. Optimization opportunities. The way in which tasks are encoded permits the system to optimize their execution automatically, allowing the user to focus on semantics rather than efficiency. Extensibility. Users can create their own functions to do special-purpose processing.
Peep - File: README
A heap inspector for live memcached instances.
High Scalability - High Scalability - Product: Scribe - Facebook's Scalable Logging System
Client Code Interface - How does your code interact with the log system? Scribe doesn't do much for you here. There's a simple Thrift interface for logging from a large set of languages, but the bulk of the work is stull up to you. Distribution System - This is were Scribe fits. It reliably (mostly) moves large numbers of messages around. A few error cases lead to data loss: 1) If a client can't connect to either the local or central scribe server the message will be loss; 2) If a scribe server crashes it could lose a small amount of data that's in memory but not on disk; 3) Some multiple component failure cases, such as a resender can't connect to any central server and its local disk fills up; 4) Some rare timeout conditions can lead to duplicate messages Do Something Usefullizer - How do you do anything useful with over 1 million messages per second? Good question. Scribe doesn't help here. But Scribe will get your data their.
Trimage (lossless) image compressor
Trimage is a cross-platform GUI and command-line interface to optimize image files via optipng, advpng and jpegoptim
Deltacloud | Many Clouds. One API. No Problem.
Start an instance on an internal cloud, then with the same code start another on EC2 or Rackspace. Deltacloud protects your apps from cloud API changes and incompatibilities, so you can concentrate on managing cloud instances the way you want.
Deltacloud Core gives you:
REST API (simple, any-platform access) Support for all major cloud service providers Backward compatibility across versions, providing long-term stability for scripts, tools and applications One level up, Deltacloud Aggregator provides a web UI in front of the Deltacloud API. With Deltacloud Aggregator, your users can:
View image status and stats across clouds, all in one place Migrate instances from one cloud to another Manage images locally and provision them on any cloud
LWPx::ParanoidAgent - search.cpan.org
The LWPx::ParanoidAgent is a class subclassing LWP::UserAgent, but paranoid against attackers. It's to be used when you're fetching a remote resource on behalf of a possibly malicious user.
Percona Server with XtraDB in Launchpad
"Percona Server with XtraDB" is a drop-in replacement for MySQL, with full backward compatibility.
Percona Server provides diagnostic tools useful to DBAs, and more tunability of server behavior. Also included is Percona's online backup utility XtraBackup, the open source tool for the XtraDB / InnoDB.
Percona Server with XtraDB shows outstanding performance and scalability on today's high-end server systems with many CPUs and fast storage subsystems such as flash storage and SSD. Notable advantages include:
Scalability: Handles more concurrent transactions, and scales up on powerful servers. Performance: Fast IO path, improved internal cooperation, and fast checksums. Flexibility: Variable page size, better table and buffer pool management, and configurable insert buffer. Reliability: Resilience to corrupted data, and crash-safe transactional replication. Management: Online backup, InnoDB table import/export, fast recovery, and configurable data dictionary. Diagnostics: Improved profiling and instrumentation.
htty, the HTTP TTY
htty is a console application for interacting with HTTP servers. It’s something of a cross between curl and the Lynx browser.
Linux sysctl configuration and tuning script | Shell Tips !
Frequently when I setup a new server (or a bunch of them) I use a small shell script that will go thru all the basic linux sysctl config. I decied to share it. I always change it a bit according to my needs and the application/services that will run on the server. Though having a script to automate some of this manual tuning is quite useful. Use it at your OWN risk. There is no silver bullet, make sure to understand each parameters and that they fit your needs/usage/network/OS.
Plugins | sfApplicationMapPlugin | 1.2.4 | symfony | Web PHP Framework
Provides you with a task generating images of appliaction-module-action structure of the project using the graphviz tool.
Introducing fise, the Open Source RESTful Semantic Engine, Nuxeo Developers Blog
As a member of the IKS european project Nuxeo contributes to the development of an Open Source software project named fise whose goal is to help bring new and trendy semantic features to CMS by giving developers a stack of reusable HTTP semantic services to build upon. As such concepts might be new to some readers, the first part of this blog post is presented as a Q&A.
CloudBees - Enterprise Java Virtualization
Introducing DEV@cloud from CloudBees
CloudBees DEV@cloud provides a fully integrated development infrastructure featuring a Hudson-based continuous integration available as a SaaS. All ready in a snap. DEV@cloud delivers:
Continuous Integration (based on Hudson) with build agents available as elastic resources billed by-the-minute, Private and secure Git and SVN repositories, Private and secure Maven repository, (more features coming soon...)
Foreman
Foreman is aimed to be a Single Address For All Machines Life Cycle Management.
Foreman integrates with Puppet (and acts as web front end to it). Foreman takes care of bare bone provisioning until the point puppet is running, allowing Puppet to do what it does best. Foreman shows you Systems Inventory (based on Facter) and provides real time information about hosts status based on Puppet reports. Foreman creates everything you need when adding a new machine to your network,It's goal being automatically managing everything you would normally manage manually - that would eventually include DNS, DHCP, TFTP, PuppetCA, CMDB and everything else you might consider useful. With Foreman You Can Always Rebuild Your Machines From Scratch! Foreman is designed to work in a large enterprise, where multiple domains, subnets and puppetmasters are required. In many cases, Foreman could help remote provisions where no experienced technicians are available.
JeOSVMBuilder - Community Ubuntu Documentation
Ubuntu JeOS (pronounced "Juice") is an efficient variant of the Ubuntu Server operating system, configured specifically for virtual appliances. It is no longer available as a CD image for download. Now you can install it with either:
the Ubuntu Server Edition installation CD by pressing F4 and choosing "Install a minimal virtual machine", or Ubuntu's vmbuilder, as described here. JeOS is a specialized installation of Ubuntu Server Edition with a tuned kernel that contains only the base elements needed to run within a virtualized environment.
MySQL DBA Tools - MySQL Expert - MySQL Consulting
The following various scripts as used for various different tasks when reviewing MySQL installations. Calculate Your MySQL Database SizeCalculate Your Individual MySQL Schema SizeLog various MySQL statisticsAnalyze MySQL statisticsGather Long/Locked MySQL SessionsAutomated MySQL Installation
Ganglia Monitoring System
Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization. It uses carefully engineered data structures and algorithms to achieve very low per-node overheads and high concurrency. The implementation is robust, has been ported to an extensive set of operating systems and processor architectures, and is currently in use on thousands of clusters around the world. It has been used to link clusters across university campuses and around the world and can scale to handle clusters with 2000 nodes.
Celery - The Distributed Task Queue
Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
The execution units, called tasks, are executed concurrently on a single or more worker servers. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).
Celery is already used in production to process millions of tasks a day.
Celery is written in Python, but the protocol can be implemented in any language. It can also operate with other languages using webhooks.
manuelpichler's php-change-coverage at master - GitHub
PHP_ChangeCoverage is an extension for PHPUnit that calculates the code coverage of thodes files that were changed during a specified time interval. This tool is useful for legacy code, where it is nearly impossible to get a good coverage for the complete code base afterwards.