Skip to content

Is it raining in Belfast? Redux

Last year I created a nano-app which attempted to answer the question, is it raining in Belfast?

I was never completely pleased with it but the objective was to get something done and get it done fast, which was achieved.

This weekend I’ve been tinkering with another project and hit a bit of a stumbling block getting various pieces of software to co-exist.

I decided a break from the problem would be beneficial but I still wanted to have some sense of achievement from the weekend and redeveloping this app seemed a good idea.

Some quick research showed that the Yahoo! weather report for Belfast could provide the necessary information so I hacked up an RSS processing script and converted the old PHP frontend to a quick and simple Sinatra app.

The app now operates as originally envisioned: it answers the question with a straightforward yes or no. Hopefully it will be accurate as well :)

The app is hosted at isitraininginbelfast.com and the code is on GitHub

2010 08 01 Ruby
Sinatra
code Comments (0) Permalink

An experimental lifestream app

Some time ago DW and myself had an enthusiastic conversation about logging various aspects of our electronic lives: emails sent and received, RSS feeds read, incoming and outgoing phone calls, the list goes on.

At that time I’d already started tinkering with apps to track my efforts in the gym and changes in my bodyweight, I’ve used notebooks for this for years and thought it’d be interesting to have this information in electronic form.

Sharpening skills

While I was on the bench waiting for my current contract to start I was sharpening my skills through developing some small projects with the technologies I’d be using. One of those technologies was SQLite which I knew of but didn’t have much hands on experience with.

I started by playing with the Ruby interface to SQLite, using it to insert and retrieve data from a simple database. I’ve since grown to love the simplicity and flexibility of SQLite and will no doubt use it again in the future.

I needed a practical application and thought back to my intentions to track my online activities. Now was as good a time as any to to tick this off the list of potential projects.

When I’d glued together some Ruby to parse the RSS feed of this blog I knew the basics were in place so I started on a simple Sinatra app to act as a web interface.

Stumbling block

I scoured the web for design inspiration and got a good start to the visual aspect of this project but couldn’t make my mind up over a few subtle items. Next thing my agent called: the purchase order finally came through and my contract was to start the next day.

The demands of adjusting to a new work environment meant I felt little motivation to work on anything on my own time and I became more interested in funneling my excess income into the stockmarket. The project joined those other half-finished unfortunates.

A fresh jolt of motivation

After a couple of months on the contract I started feeling a renewed motivation to get something of my own out there. I couldn’t bare the thought of starting another project to not finish it so I set out to to whip this latest app into shape and release it as soon as possible and no later.

I dug through the uncommited changes in my working copy and made some arbitrary decisions which resulted in a design I felt was “good enough.” I modified my RSS parsing script to feed into SQLite and extended it to process my Delicious bookmarks and loved tracks on Last.fm. I created pagination links, wrote a README and sorted out the hosting and cron jobs. Bingo.

Check it out

My original intention was to track much more of my online life and there’s some details I’d like to change but I love releasing code and I’m pleased with this first iteration.

The app is hosted at life.stevenwilkin.com and the code is available on GitHub, I hope you like it.

2010 06 27 Ruby
Sinatra
Web
code Comments (5) Permalink

Rails 3 Hello World

The past while I’ve been busy providing technical consultancy to BT and haven’t had much drive to work on anything on my own time. The itch has returned the past couple of weeks though so I thought I’d see what I’ve been missing in the Rails world and in what better way than getting a basic Rails 3 app up and running.

My environment was already setup for Rails 2.3.* and Yehuda Katz’ post served as a guide to get me up to speed with the beta loveliness.

RubyGems 1.3.7 along with Thor and Bundler gems required

My installed version of RubyGems was a couple of point releases behind, so I updated that and installed the necessary gems

sudo gem update --system
sudo gem install thor bundler

Clone Edge Rails from GitHub

mkdir -p ~/code/rails/rails
git clone http://github.com/rails/rails.git ~/code/rails/rails

Generate a fresh app and install dependencies with Bundler

mkdir ~/code/rails/rails-3-demo
cd !$
ruby ~/code/rails/rails/bin/rails new . --dev
bundle install

Launch the web server

./script/rails server

Browse to http://0.0.0.0:3000 and you’re done!

2010 06 08 Rails
Ruby
code Comments (0) Permalink

Home time revisited – redeveloping a CakePHP application with Sinatra

I’ve just released the latest iteration of hometi.me, a little nano-app I’ve mentioned before.

Deliberate Practice

It’s only a trivial app but I’ve a bit of free time before the start of a contract so I thought it would be good practice to redevelop it, a task I’d been ignoring for a long time.

Goodbye CakePHP, Hello Sinatra

The original app was put together quickly with CakePHP, a PHP-based framework I’ve used since my first paid programming gig. Since then though my interest and proficiency with Ruby and it’s frameworks has come more to the forefront. By choosing Sinatra, the app was given access to the rich Ruby ecosystem, including the joyful HAML and SASS.

The interesting aspect of this release is that all the logic is now implemented on the client-side using Javascript. Originally the countdown was calculated on the server-side and passed to the client with an AJAX call, but the server is more-or-less just hosting the markup, stylesheets and Javascript files required to get the app up and running in the browser.

Heroku

As this app doesn’t make use of a database or Cron jobs etc I thought I’d take the opportunity to move it from my own infrastructure and onto the awesomeness of Heroku whose free offering is a lot more performant than my little VPS. Heroku is a service I’ve been playing with for a while now and it makes deploying a Ruby web app into production a breeze, assuming your project can work within it’s limitations.

All that’s left now is for the DNS changes to propagate. I’d like to show a bit more love to this app in the future, the controls to specify your own home time could be more intuitive for instance, but for now I’m content with the improvements I’ve made to it and it’s new home.

Is it passed your home time yet?

2010 03 24 Ruby
Sinatra
Web
code
javascript Comments (0) Permalink

node-shoutbox – a simple node.js demo

Last month, at the inaugural Super Happy Dev Castle, my buddy Dave Rice told me about a project he was working on involving node.js, an event-based I/O system for Javascript.

Javascript outside of the browser?

This concept seemed almost ridiculous to me at that time but I thought it was worth a look and cobbled together a quick “Hello World.” I was greatly amused by creating a command-line app with Javascript but did nothing more about it until last week. I’m currently starting out as an independant contractor & have a bit of spare bandwidth so I thought I’d put together something a bit meatier in order to learn more about this system & to also sharpen up my Javascript skills.

Dave had shown me node-comment and I also studied node_chat. From there I had just enough reference material to understand the mechanics of long-polling and to start piecing together a simple real-time commenting system.

What is node?

In and of itself, node is not a web development stack. It provides an I/O platform of which HTTP handling is a part. People are building blogging and wiki engines, FTP servers and command-line tools with this thing, anything I/O based is possible and thankfully micro-frameworks like (fab) can ease the pain of creating web apps.

Node is still very young software, which I have felt the pain of. To serve up static content I wanted to use paperboy as it can be used as middleware within (fab), however mere days after the last update to the project the API it was relying upon in node changed fundamentally. Despite being new to node development I was able to patch paperboy to work with node v0.1.31 and get on with my actual project. Another sticking point was the inability of (fab) to handle POSTed form variables, this functionality is planned for it’s next release though.

Fun

Ultimately, my app is of little practical use: it doesn’t make use of persistance or moderation and node’s suitability for the production environment may itself be questionable at this time, but it was quite a fun experience. From something I more-or-less despised a decade ago I’ve come to develop a real affection for coding Javascript over the past few years.

When I had the basics of the app laid-down I eagerly awaiting the moment when I could have it open in two browser windows and enter a comment into one and see it appear in the second. That moment came and it brought a smile to my face. I suggest you give it a go :)

Get the code

As a wise man once said, code speaks louder than words, so please be my guest and clone my demo on GitHub.

2010 03 19 Web
code
javascript Comments (0) Permalink

A dummy datasource for CakePHP

I was recently developing a project at work which at the beginning didn’t need a database. However, Cake was still testing the database connection and issuing a warning as a result when it couldn’t connect. My solution was to create a dummy datasource which silenced the warnings and allowed me to get on with the rest of development in relative peace.

The scope of the project changed and a database was required. The dummy datasource had to go so I thought I’d share it here before it was thrown to the winds.

Here’s the datasource, which belongs in models/datasources/dbo/dbo_dummy.php:

<?php
class DboDummy extends DboSource {
 
        function connect(){
                $this->connected = true;
                return true;
        }
 
        function disconnect(){
                $this->connected = false;
                return true;
        }
 
}
?>

To use the datasource use the following in your config/database.php:

<?php
class DATABASE_CONFIG {
 
        var $default = array(
                'driver' => 'dummy',
                'host' => '',
                'login' => '',
                'password' => '',
                'database' => '',
        );
 
}
?>

Enjoy!

2010 02 14 PHP
code Comments (2) Permalink

Migrating a WordPress database between domains

Occassionally I find myself developing a WordPress theme which will then require moving from development into production or otherwise having to move a blog between domains.

Apart from the transfer of the files, including plugins, theme and core WordPress installation, there is only 1 slightly gotcha: the database. I’ve never been quite sure why but WordPress stores the URL of the blog in it’s database, not once, but twice. For the blog to be migrated the database needs a slight tweak.

As always, create a backup before doing anything. This can be done with phpMyAdmin or using the mysqldump command like so:

$ mysqldump -uUSER -pPASSWORD DATABASE > /path/to/backup.sql

If the database is to be hosted on a different machine you can then import this dump using using which ever method you’re comfortable with. Finally, run the following query on the database, subsituting your own domain name:

UPDATE
	wp_options
SET
	option_value = 'http://DOMAIN.TLD'
WHERE
	option_name IN ('siteurl', 'home')

That’s it!

2010 01 05 MySQL
Tech Comments (2) Permalink

MySQL database backup with remote storage

Prevent a disaster

After reading Jeff Atwood’s backup failure last month I decided to finally get around to doing something I’d been intending to do “one of these days” but had in actual fact been putting off for years.

Here’s the steps I took to ensure the databases on my webserver were backed up every night and copies of the dumps stored remotely.

On the remote storage machine

Generate an ssh key pair with and empty password and put the public key on the remote server. This will give our script access to the server without requiring you to enter a password each time:

$ ssh-keygen -t rsa -f /home/steve/code/db_backup/id_rsa
$ scp /home/steve/code/db_backup/id_rsa.pub REMOTEHOST:

This script will fetch all the backups, logging in as the rsync user and using the private key just generated. It’s located at /home/steve/code/db_backup/sync_backups.sh:

#!/usr/bin/env bash
rsync -e "ssh -l rsync -i /home/steve/code/db_backup/id_rsa" -avz REMOTEHOST:mysql/ /data/primary/backup/mysql/

Have this happen automatically daily at 12:20am:

$ crontab -l
# m h dom mon dow command
20 0 * * * /home/steve/code/db_backup/sync_backups.sh
$

On the machine to be backed up

Create a new user and allow ssh access with the previously generated key:

# adduser rsync
# mkdir ~rsync/.ssh
# mv ~steve/id_rsa.pub ~rsync/.ssh/authorized_keys
# chown rsync:rsync ~rsync/.ssh/authorized_keys
# chmod 400 ~rsync/.ssh/authorized_keys

This script will dump all available databases and is located at /root/bin/backup_databases.sh:

#!/usr/bin/env bash
 
# dump all available databases
# SJW
 
AUTH='-uroot -pROOTPASSWORD'
DBS=`mysql $AUTH --skip-column-names -e 'SHOW DATABASES;'`
BACKUPS='/home/rsync/mysql/'
 
for DB in $DBS
do
	mysqldump $AUTH $DB > $BACKUPS`date +%Y%m%d%H%M`_$DB.sql
done
 
# delete backups older than 5 days
find $BACKUPS -mtime +5 -type f | awk '{print "rm "$1}' | sh

Have the script run nightly at 12:10am via cron:

# crontab -l
# m h dom mon dow command
10 0 * * * /root/bin/backup_databases.sh
#

Closing thoughts

This approach is realtively straight forward, everything happens automatically and it could easily be extended to cover mailboxes, source code repositories, uploaded content etc. However, for mission-critical databases master-slave replication may be more appropriate. For further reading you may enjoy JWZ’s thoughts on backups.

2010 01 04 Linux
Tech Comments (2) Permalink

6 months of Rehab

I find it hard to believe, but I crossed the 6 month milestone at Rehab this week.

The time has went in quickly, with a fast-pace and quick turnaround times being the norm. The vibe in the office has been great, with a decent soundsystem pumping out dance music all day and my producer Gary, aka DJ Gary Quinn, dropping in a few choice chunes.

The guys here have a real passion for the web and enjoy a good laugh, for instance, this week we had a challenge to wear a different hat each day. Here’s Iain and Jurgen in their full cowboy glory:

I spend most of my days wrangling CakePHP, the PHP framework I started developing with nearly 3 years ago now. I also do most of the administration of the Linux boxes used to host the client sites and automate as much as I can with handy bash scripts, all from the comfort of this shiny iMac:

On the downside the standard working hours are 9 to 6, which is longer than I’m accustomed to. You can’t have it all ;)

2009 12 13 Clerical
Photo Comments (4) Permalink

Mashing up Twitter Trends and Google Images

I recently released the latest entry in my series of nano-apps.

As with my previous efforts, istrending.com serves serves no real purpose other than as an act of deliberate practice: going through the process of coming up with an idea, writing code, registering domain names, setting up Cron jobs and configuring deployment systems to get it all up and running in a production environment.

I questioned why I was even bothering with an app that was likely not going to see any traffic or be of use to people and when I hit a minor javascript problem the project was shelved for a few months. I recently had a fresh burst of enthusiasm though and got the usual sense of achievement when I deemed the code “ready enough” and a quick cap deploy took care of the rest.

The main problem I’ve faced with this app concerns the script used to update the list of trending topics: the Twitter search api has a known issue of returning HTML instead of JSON, which has on occasion caused my inbox to be filled with emails from the Cron daemon alerting me to failed jobs. I’m also not very happy with how the javascript carousel functions, so there’s some scope for improvement there.

The code is available on GitHub as usual.

Next up is probably something bigger, very likely involving Rails. I’ve been reading up on the various testing methodologies used in the Ruby world and have written my first set of RSpec tests so I hoping my next release will at least have some test coverage. Other than that I’ll just have to see what ideas come to mind between now and then.

Any suggestions on what I should do next?

2009 09 15 Ruby
Sinatra
code Comments (0) Permalink