Build ImageMagick from source

curl -O ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick.tar.gz
tar xvzf ImageMagick.tar.gz -C /usr/src/
cd /usr/src/ImageMagick-6.6.6-6/
./configure
make
make install

Install the RMagick Gem

gem install rmagick

Bingo!

As an interesting aside, ImageMagick handles a lot of image formats via delegate libraries. I previously installed RMagick on a CentOS box and had to seperately install TrueType fonts which were necessary for the project in question. These had to be installed before ImageMagick and were accessible through yum and the xorg-x11-fonts-truetype package.

A few months ago I was tasked with migrating and maintaining a bunch of legacy Rails apps and a couple of them were misbehaving. Requests were hanging which was tying-up the front-end Apache child processes and resulting in all the web-apps on the server becoming unresponsive

At the time I didn’t know what was causing the hanging requests, it wasn’t happening in a predictable manner and on the surface I had all the apps dependencies in place.

Seeing as this was a job I was doing on the side I had very limited time to investigate the underlying issue and decided I would have to make do with just knowing when the apps went down so I could ssh in and manually kill off the offending processes.

My initial thought was to use Nagios for monitoring but it seemed excessively heavy-weight for the task at hand so I seized the opportunity to quickly develop a tool perfectly tailored to my needs.

The resulting script, minder, checks a list of domains and if the root of each can’t be read within 10 seconds a notication is sent via XMPP. Nothing more, nothing less. Simple.

Minder proved effective and I was bombarded with instant messages whenever any of the apps went under. Thankfully I found to bit more time to investigate more thoroughly and tracked the issue down to a missing TrueType font used by ImageMagick.

The first iteration of minder was very simple but I thought I’d put a bit of extra effort in to make it easier to customise and share it with the internet-at-large, hopefully someone will get some use out of it.

Prerequisties

• Two XMPP accounts:

  • one to send notifications from, and
  • a second to receive notifications

  Google Talk or any regular Jabber/XMPP accounts will suffice

• Blaine’s xmpp4r-simple gem must be installed:

  sudo gem install xmpp4r-simple
  

Usage

• copy minder.yaml.sample to minder.yaml
• edit minder.yaml to specify the username and password of the sending XMPP account
• specify your own personal XMPP account (xmpp_to)
• modify the list of domains to monitor
• run the script

  $ ./minder.rb
  

• Optionally run the script from a Cron job, the following will run every hour:

  0 * * * * /path/to/minder.rb > /dev/null 2>&1
  

Get the code

Clone the repo from GitHub. Fire any queries to @stevebiscuit.

After reading Jeff Atwood’s backup failure last month I decided to finally get around to doing something I’d been intending to do “one of these days” but had in actual fact been putting off for years.

Here’s the steps I took to ensure the databases on my webserver were backed up every night and copies of the dumps stored remotely.

On the remote storage machine

Generate an ssh key pair with and empty password and put the public key on the remote server. This will give our script access to the server without requiring you to enter a password each time:

$ ssh-keygen -t rsa -f /home/steve/code/db_backup/id_rsa
$ scp /home/steve/code/db_backup/id_rsa.pub REMOTEHOST:


This script will fetch all the backups, logging in as the rsync user and using the private key just generated. It’s located at /home/steve/code/db_backup/sync_backups.sh:

#!/usr/bin/env bash
rsync -e "ssh -l rsync -i /home/steve/code/db_backup/id_rsa" -avz REMOTEHOST:mysql/ /data/primary/backup/mysql/

Have this happen automatically daily at 12:20am:

$ crontab -l
# m h dom mon dow command
20 0 * * * /home/steve/code/db_backup/sync_backups.sh
$


On the machine to be backed up

Create a new user and allow ssh access with the previously generated key:

# adduser rsync
# mkdir ~rsync/.ssh
# mv ~steve/id_rsa.pub ~rsync/.ssh/authorized_keys
# chown rsync:rsync ~rsync/.ssh/authorized_keys
# chmod 400 ~rsync/.ssh/authorized_keys


This script will dump all available databases and is located at /root/bin/backup_databases.sh:

#!/usr/bin/env bash
 
# dump all available databases
# SJW
 
AUTH='-uroot -pROOTPASSWORD'
DBS=`mysql $AUTH --skip-column-names -e 'SHOW DATABASES;'`
BACKUPS='/home/rsync/mysql/'
 
for DB in $DBS
do
	mysqldump $AUTH $DB > $BACKUPS`date +%Y%m%d%H%M`_$DB.sql
done
 
# delete backups older than 5 days
find $BACKUPS -mtime +5 -type f | awk '{print "rm "$1}' | sh

Have the script run nightly at 12:10am via cron:

# crontab -l
# m h dom mon dow command
10 0 * * * /root/bin/backup_databases.sh
#


Closing thoughts

This approach is realtively straight forward, everything happens automatically and it could easily be extended to cover mailboxes, source code repositories, uploaded content etc. However, for mission-critical databases master-slave replication may be more appropriate. For further reading you may enjoy JWZ’s thoughts on backups.

I had intended to shutdown this machine since the start of the year but never got around to it and after a techie chat on IM with Dave Dripps earlier in the day decided to just “pull the finger out” and do the needful.

I archived the contents of /var/www/htdocs and my home directory for safe keeping, copying them over to my file server, and dumped all the MySQL databases I still hadn’t migrated.

Self-hosting a site was a great learning opportunity but not something a business could be built up upon and comparing the energy cost of running a machine 24×7 versus renting a VPS priced in American Dollars the choice wasn’t hard to make, it’s just a shame it took me half a year to finally flick the switch.

Goodnight substance!

Some additional information is available on the BLUG Facebook group and on the Belfast Linux newsgroup.

See you there!

steve@decaf:~$ history | awk '{a[$2]++} END {for(i in a)print a[i] " " i}' | sort -rn | head -10
190 ls
80 cd
24 cp
22 sudo
19 rm
14 svn
12 history
11 tar
10 wget
10 vi

And as root:

172 ls
59 vi
56 cd
22 apt-get
20 less
20 apache2ctl
17 apt-cache
15 cp
10 pwd
10 ps

I’m kinda suprised that ls comes out on top each time

I wonder which commands Matt, Aidan and Philip have been using most?

Well, the good news is that this technique works

All it took was 3 simple steps:

1. $ cd /var/www/sickbiscuit.com/blog
2. $ svn switch http://svn.automattic.com/wordpress/tags/2.5/ .
3. launch wp-admin/upgrade.php via web-browser

To be safe I backed up the database prior to the update and so far everything seems good, job’s a good ‘un!

Today I was talking to Matt and he mentioned updating one of his WordPress installations and I noticed I was due an update myself. I downloaded the latest release and was having a quick skim through the upgrade procedure to make sure I wasn’t forgetting about anything and I spotted a link to the Subversion update instructions… I’m off work sick today and have the time so I decided to give it a go.

I backed up my database and checked out the latest stable version:

steve@decaf~$ cd /var/www/sickbiscuit.com
steve@decaf:/var/www/sickbiscuit.com$ svn co http://svn.automattic.com/wordpress/tags/2.3.3/

I copied over my database config file, theme, plugins and uploads and ran the wp-admin/upgrade.php script via my browser. The final act was to modify the symlink pointing to the WordPress directory and it worked first time. Profit!

Hopefully whenever I need to update in the future all I’ll have to do is use the following:

svn sw http://svn.automattic.com/wordpress/tags/NEW_VERSION/

followed by running the database upgrade script again. Quick and painless.

DNS records have been updated and decaf is now handling mail and web traffic for the domain allow the only thing I’ve copied over is this blog.

Hopefully this will give me the motivation needed to spruce things up a bit as the last iteration of sickbiscuit.com looked like it was designed by a programmer

I had been considering this for a while, especially after my experience using an Ubuntu VPS with Infurious and after 2 power failures within as many weeks due to building work nearby to my home, my hand was forced. No more hosting on a Linux box on the end of a DSL connection for me!

I opted for a XEN based VPS running Debian Etch. I've really come to love APT based distros after running Kubuntu on my desktop before I was endowed with a Mac and with the relative ease of setting up all the Infurious services on Ubuntu. So I decided to go upstream and it's a far cry from my past experiences with Slackware

My first priority was getting my LAMP stack up and running and I spent my free time over the past few days following this excellent tutorial.

Just like many other things in the FLOSS world: you get the instructions, follow those instructions and it Just Works. This instance was no different and all I really had to so was copy and paste commands & configuration settings and I probably spent more time doing background reading, testing each part as I went along and keeping track of all the changes I made on my personal wiki.

The result is I can now host email accounts for as many domains I wish, provide access to those accounts over IMAPS and perform server-side virus scanning and spam filtering.

I've said it before and I'll say it again: I love free software!