
Archives (2008 - 2023)
The following various scripts as used for various different tasks when reviewing MySQL installations. Calculate Your MySQL Database SizeCalculate Your Individual MySQL Schema SizeLog various MySQL statisticsAnalyze MySQL statisticsGather Long/Locked MySQL SessionsAutomated MySQL Installation
Thanks to the devopsdays conference, the idea of devops seems to live on. While talking with other people about it, I realize that it is difficult to frame it within the current IT landscape. At lot of the ideas are coming from different kinds of emerging technologies (T) and process management (P) approaches.
For me the two most important observations are:
there is a increase in feedback loops between business, all parts of the delivery process and operations thanks to this feedback loops we increase the quality and speed up the flow So where can you look for devops ideas ? As you can see on the map , these interactions are all over the place.
Ubuntu JeOS (pronounced "Juice") is an efficient variant of the Ubuntu Server operating system, configured specifically for virtual appliances. It is no longer available as a CD image for download. Now you can install it with either:
the Ubuntu Server Edition installation CD by pressing F4 and choosing "Install a minimal virtual machine", or Ubuntu's vmbuilder, as described here. JeOS is a specialized installation of Ubuntu Server Edition with a tuned kernel that contains only the base elements needed to run within a virtualized environment.
Foreman is aimed to be a Single Address For All Machines Life Cycle Management.
Foreman integrates with Puppet (and acts as web front end to it). Foreman takes care of bare bone provisioning until the point puppet is running, allowing Puppet to do what it does best. Foreman shows you Systems Inventory (based on Facter) and provides real time information about hosts status based on Puppet reports. Foreman creates everything you need when adding a new machine to your network,It's goal being automatically managing everything you would normally manage manually - that would eventually include DNS, DHCP, TFTP, PuppetCA, CMDB and everything else you might consider useful. With Foreman You Can Always Rebuild Your Machines From Scratch! Foreman is designed to work in a large enterprise, where multiple domains, subnets and puppetmasters are required. In many cases, Foreman could help remote provisions where no experienced technicians are available.
Introducing DEV@cloud from CloudBees
CloudBees DEV@cloud provides a fully integrated development infrastructure featuring a Hudson-based continuous integration available as a SaaS. All ready in a snap. DEV@cloud delivers:
Continuous Integration (based on Hudson) with build agents available as elastic resources billed by-the-minute, Private and secure Git and SVN repositories, Private and secure Maven repository, (more features coming soon...)
While much attention has been focused on high-level software architectural patterns, what is, in effect, the de-facto standard software architecture is seldom discussed. This paper examines this most frequently deployed of software architectures: the BIG BALL OF MUD. A BIG BALL OF MUD is a casually, even haphazardly, structured system. Its organization, if one can call it that, is dictated more by expediency than design. Yet, its enduring popularity cannot merely be indicative of a general disregard for architecture.
These patterns explore the forces that encourage the emergence of a BIG BALL OF MUD, and the undeniable effectiveness of this approach to software architecture. What are the people who build them doing right? If more high-minded architectural approaches are to compete, we must understand what the forces that lead to a BIG BALL OF MUD are, and examine alternative ways to resolve them.
A number of additional patterns emerge out of the BIG BALL OF MUD. We discuss them in turn. Two principal questions underlie these patterns: Why are so many existing systems architecturally undistinguished, and what can we do to improve them?
As a member of the IKS european project Nuxeo contributes to the development of an Open Source software project named fise whose goal is to help bring new and trendy semantic features to CMS by giving developers a stack of reusable HTTP semantic services to build upon. As such concepts might be new to some readers, the first part of this blog post is presented as a Q&A.
The Ruby developers (or expert users) community is generally very hostile towards Debian. Many harsh words and insults are used in discussions mentioning Debian in the ruby-core@ mailing list, and there are frequent recommendations to avoid the Debian packages and install from sources, which is quite demotivating (Actually, what prompted that blog post is someone calling Debian-specific changes unforgivable).
This atmosphere makes it hard to recruit people, and the Debian Ruby teams are completely understaffed, which is clearly the major blocker to improving the situation further
"Percona Server with XtraDB" is a drop-in replacement for MySQL, with full backward compatibility.
Percona Server provides diagnostic tools useful to DBAs, and more tunability of server behavior. Also included is Percona's online backup utility XtraBackup, the open source tool for the XtraDB / InnoDB.
Percona Server with XtraDB shows outstanding performance and scalability on today's high-end server systems with many CPUs and fast storage subsystems such as flash storage and SSD. Notable advantages include:
Scalability: Handles more concurrent transactions, and scales up on powerful servers. Performance: Fast IO path, improved internal cooperation, and fast checksums. Flexibility: Variable page size, better table and buffer pool management, and configurable insert buffer. Reliability: Resilience to corrupted data, and crash-safe transactional replication. Management: Online backup, InnoDB table import/export, fast recovery, and configurable data dictionary. Diagnostics: Improved profiling and instrumentation.
Start an instance on an internal cloud, then with the same code start another on EC2 or Rackspace. Deltacloud protects your apps from cloud API changes and incompatibilities, so you can concentrate on managing cloud instances the way you want.
Deltacloud Core gives you:
REST API (simple, any-platform access) Support for all major cloud service providers Backward compatibility across versions, providing long-term stability for scripts, tools and applications One level up, Deltacloud Aggregator provides a web UI in front of the Deltacloud API. With Deltacloud Aggregator, your users can:
View image status and stats across clouds, all in one place Migrate instances from one cloud to another Manage images locally and provision them on any cloud
Client Code Interface - How does your code interact with the log system? Scribe doesn't do much for you here. There's a simple Thrift interface for logging from a large set of languages, but the bulk of the work is stull up to you. Distribution System - This is were Scribe fits. It reliably (mostly) moves large numbers of messages around. A few error cases lead to data loss: 1) If a client can't connect to either the local or central scribe server the message will be loss; 2) If a scribe server crashes it could lose a small amount of data that's in memory but not on disk; 3) Some multiple component failure cases, such as a resender can't connect to any central server and its local disk fills up; 4) Some rare timeout conditions can lead to duplicate messages Do Something Usefullizer - How do you do anything useful with over 1 million messages per second? Good question. Scribe doesn't help here. But Scribe will get your data their.
Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets.
At the present time, Pig's infrastructure layer consists of a compiler that produces sequences of Map-Reduce programs, for which large-scale parallel implementations already exist (e.g., the Hadoop subproject). Pig's language layer currently consists of a textual language called Pig Latin, which has the following key properties:
Ease of programming. It is trivial to achieve parallel execution of simple, "embarrassingly parallel" data analysis tasks. Complex tasks comprised of multiple interrelated data transformations are explicitly encoded as data flow sequences, making them easy to write, understand, and maintain. Optimization opportunities. The way in which tasks are encoded permits the system to optimize their execution automatically, allowing the user to focus on semantics rather than efficiency. Extensibility. Users can create their own functions to do special-purpose processing.
One of the great difficulties of Facebook application development is the difficulty in running large-scale tests due to the large number of Facebook accounts required, and the Facebook Terms of Use which does not allow automated systems to access the Facebook servers. These issues are explained in depth in Load testing Facebook apps (Part 1) and Load testing Facebook apps (Part 2). FriendRunner acts as a drop-in replacement for the Facebook servers, and so allows you to run your tests with as many users as you'd like. Furthermore, FriendRunner's automated test system runs the entire test for you once you create a test script to define the actions for a single user.