Found 8 bookmarks
Newest
How Twitter Uses NoSQL - ReadWriteCloud
How Twitter Uses NoSQL - ReadWriteCloud
Weil is quick to point out that Twitter is heavily dependent on MySQL. However, Twitter does employ NoSQL solutions for many purposes for which MySQL isn't ideal. According to Weil, Twitter users generate 12 terrabytes of data a day - about four petabytes per year. And that amount is multiplying every year. Read on for our notes on Weil's talk.
How Twitter Uses NoSQL - ReadWriteCloud
redisql - Project Hosting on Google Code
redisql - Project Hosting on Google Code

Redisql is a lightweight SQL server AND Redisql is built on top of the NOSQL datastore redis, supports redis data-structures and redis commands and supports (de)normalisation of these data structures (lists,sets,hash-tables) to/from SQL tables. Redisql can also easily import/export tables to/from Mysql for Data-warehousing. Redisql is not only a data storage Swiss Army Knife, it is also extremely fast and extremely memory efficient.

Speed is achieved by being an event driven network server that stores ALL data in RAM and achieves disk persistence by using a spare cpu-core to periodically log data changes (i.e. no threads, no locks, no undo-logs, serving data over a network at RAM speed) Storage data structures w/ very low memory overhead and data compression, via algorithms w/ insignificant performance hits, greatly increase the amount of data you can fit in RAM Your hard disk's swap is utilised when your data can no longer fit in RAM. In this mode, performance is not negatively effected, if rarely-used data sits idle in swap. Redisql can use 100% of your RAM for storage and still provide disk persistence. Optimising to the SQL statements most commonly used in OLTP workloads yields a lightweight SQL server designed for low latency at high concurrency (i.e. mindblowing speed).

redisql - Project Hosting on Google Code
The Kumofs Project
The Kumofs Project

Kumofs is a simple and fast distributed key-value store. You can use a memcached client library to set, get, CAS or delete values from/into kumofs. Backend storage is Tokyo Cabinet and it will give you great performance. Data is partitioned and replicated over multiple servers. Extreme single node performance; comparable with memcached. Both read and write performance got improved as servers added. Servers can be added without stopping the system. Servers can be added without modifying any configuration files. The system does not stop even if one or two servers crashed. The system does not stop to recover crashed servers. Automatic rebalancing support with a consistency control algorithm. Safe CAS operation support. memcached protocol support. Kumofs is used at Nico Nico douga (Wikipedia), the most popular video sharing service in Japan.

The Kumofs Project
Gemini Mobile Technologies
Gemini Mobile Technologies

Hibari is a distributed, non-relational database management system (distributed non-RDBMS) based on distributed key value store technology, implemented by Gemini Mobile Technologies, it inherits important characteristics of both the distributed non-RDBMS world and of Gemini's HyperScale® Platform.

Hibari has the following characteristics to flexibly support growing volumes of data in cloud computing space. Economic system build-up using commodity hardware Flexible, high scalability High performance and availability by consistent hashing across multiple machines High fault-tolerance by chain replication across multiple machines Safety without a single point of failure (SPOF)

Gemini Mobile Technologies
Home - stalkr - GitHub
Home - stalkr - GitHub

Stalkr is a data persistence library. Stalkr enables your PHP web application to interact with database servers in a fully object-oriented pattern. No more messy SQL, no more nasty injection attacks!

This type of library is usually called an object-relational mapper, or ORM for short. However, Stalkr is different from your typical ORM. In addition to the usual “mapping” between PHP objects and SQL rows, Stalkr also boasts a wealth of scalability-friendly features, such as:

Exclusive features (of sorts: see the Yet Another ORM!? wiki page for more information)

Automatic and transparent integration with Memcached (drivers included!) Support for key-value stores, with built-in consistent hashing and redundancy (now with Redis support!) Seamless utilization of multiple clusters of potentially different database software Unique access control mechanism for additional security

Home - stalkr - GitHub
The Apache Cassandra Project
The Apache Cassandra Project

The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together Dynamo's fully distributed design and Bigtable's ColumnFamily-based data model.

Cassandra was open sourced by Facebook in 2008, where it was designed by Avinash Lakshman (one of the authors of Dynamo) and Prashant Malik. In a lot of ways you can think of Cassandra as Dynamo 2.0. Cassandra is in production use at Rackspace, Digg, and a number of other companies, but is still under heavy development.

The Apache Cassandra Project