I came home from work on Monday and noticed it was just showing a background wallpaper, no mouse response or such like so I resorted to a hard reboot. On rebooting I was presented with the grey boot background and an image of a flash folder – uh oh! My first Mac boot failure. Now I wouldn’t have been concerned as I use Mozy an online backup tool (a bargain at $4.95 for unlimited backup) but in this case I’d just added a load of videos from my wedding and honeymoon and knew that it was taking a while to backup – one downside of online backup. I also make heavy use of DropBox with symbolic linking so important stuff like bash customisations, ssh keys etc are shared between my machine as well as various project directories so I knew these were all safe.

I booted from the Snow Leopard CD (holding down the C key) to run the disk utility but a verify/repair didn’t work and kept returning ‘Invalid Node Structure’ message. Uh oh! Then I remembered about ‘Target Disk Mode’ using a Firewire cable between two Macs and holding down the ‘T’ key on boot – all this let me connect to was my Vista boot camp partition – ah, so I could browse that but not the OSX partition. I figured out that if it were a hardware problem the Vista partition wouldn’t have booted, but it did.

Still getting nowhere I did a bit of Googling and eventually found software that enabled me to see the contents of the OSX partition so I hurried out and got myself an external 1Tb drive so I could copy the data off resound to the fact I’d be rebuilding this machine – which again wasn’t really a problem since this Mac was originally Tiger, then upgraded to Leopard and subsequently Snow Leopard.

After rebuilding the machine the drive I’d previously purchased was immediately enabled as a Time Machine backup volume, moved the data back from the disk to where it should be on the computer and reinstalled Mozy (after contacting technical support to find out what it would do since it would already have the data backed up and I haven’t restored it from them).

So now reflecting back on this scenario I’ve come to a number of conclusions. Firstly, backup, backup, backup.

• Online backup tools like Mozy, Carbonite are great to get your most important and valuable data (I’m thinking primarily Photos and Videos) out of your house but are severely limited by upload speeds and presumably download speed if you’ve got a lot of data stored to retrieve.
• DropBox is great for smaller content, documents, code repositories (my Rails projects using GIT are in a DropBox folder, no need to remote sync to locations like GitHub)
• Local backup is still essential, getting data out of your house covers you if your house burns down or such like but hardware failure is much more likely – if you’re not already get yourself an external drive to use as a Time Machine vault. I noticed that you can restore a machine from Time Machine backup during the rebuild process – this would have saved me significant time and effort had I already been using Time Machine.
• Lastly, OSX isn’t as rock solid as I thought. There are plenty of ‘Invalid Node Structure‘ results on Google but I couldn’t find any definitive answer/solution to the problem.

Hope that helps someone…

The Chancellor of the UK has in his wisdom today announced/confirmed in his pre-budget announcements that a new ‘Broadband Tax’ will be introduced that will be used to;

….fund the installation of “super-fast” broadband to 90% of the population by 2017.

It will be £6 a year and everyone in the UK with a land line will have to pay the tax. WTF? That’s pretty much 99% of the population I’d imagine aiming to raise around £150m a year to fund the roll out.

So, let me get this straight – The government is taxing me to pay a private company (British Telecom) to roll out fast broadband to the population???? I already have fast broadband thanks, I saw the light and left BT’s crappy antiquated copper crap and went to Virgin Media where I really do enjoy 10Mb of speed and can wind it up to 50Mb if I so choose.

It’s also to fund roll out of broadband to ‘rural areas’ – does the government not think that perhaps people move to rural areas to get away from modern tech? Broadband has been rolling out quite steadily over the past years and yes, maybe the UK is behind the rest of the world in speeds but haven’t we got more pressing issues to worry ourselves with like the mess the bankers got this country in and the massive bailouts I’m we are already paying for!!

The good news is the conservatives say they would scrap this ‘legislation’ if they got into power – Labour your days are numbered!

We have all our CFCs implemented using CFSCRIPT as CF9 has provided us the ability to do and we had a situation where we needed to perform a SQL query with an IN clause. In tags that’s a pretty routine query but in script we kept getting errors.

queryservice.setName("users");
queryservice.setdatasource(get('datasourcename'));
queryservice.addParam(name="usergroups",value=params.usergroups,list="yes");
result = queryservice.execute(sql="select email from users where user_group_id in (:usergroups) ");
writeDump(result.getResult());

Seems pretty straightforward right? We knew that params.usergroups may be a list so we’d accounted for that in the addparam – it’s a series of checkboxes for the user to select using this syntax we got the error message returned;

Named Sql parameter ‘usergroups)’ used in the query not found in the queryparams

Well, after some more head scratch we found the problem – when you’re using named parameters using a notation similar to symbols in Ruby with brackets in the SQL statement they need to have a space AFTER the name of the parameter! So (:usergroups) becomes (:usergroups ) – notice the extra space.

Hope that helps someone!

Most browsers (if they comply with the RFC) will open a maximum of 4 HTTP connections to a single URL, that is to say that it is able to load 4 assets simultaneously from a URL – this behaviour is easy to see if you use the Net panel of Firebug, you see the assets ’stair stepping’ as they load.

Now the easiest way to boost your page load time is to create (assuming you have the necessary access to the server) a new virtual server pointing at your assets folder and have it setup with host headers like assets0.yourdomain.com, assets1.yourdomain.com and then ensure that your assets load from http://assets0.yourdomain.com etc…but this still means that it’s coming from the same server.

Enter Amazon S3. It’s marketed as ’storage for the internet’. It’s cheap storage with cheap access rates which can be public or private – to give you some idea of price I store about 25Gb of photos up there and my last bill was about $3 – granted those photos are not accessed so I’m not paying for GET requests since I’m not requesting the data but I sleep soundly knowing all my photos are safely backed up out of my house (I use JungleDisk on my Mac/PC to perform this backup for me!) Anyhow, I digress.

Amazon S3 can be accessed in a variety of ways – the simplest is to use the S3FOX plugin for Firefox. With S3 you start off creating a ‘bucket’. Whilst you can name a bucket anything, it’s best to keep it globally unique using a FQDN, such as assets0.mydomain.com. Amazon will let you create a none globally unique bucket name but behind the scenes it would have been made unique. There are infact different rules for buckets stored in the EU, these must be DNS compatible

Buckets by default are private along with anything you place in the bucket – to allow the assets to be public you need to edit the ACL for the bucket and allow ‘Everyone’ read access and apply this to the subfolders too. Assuming you’ve done that then your assets would now be accessible via http://s3.amazonaws.com/assets0.mydomain.com/ and also via http://assets0.mydomain.com.s3.amazonaws.com/.

But it doesn’t stop there – assuming you’ve setup a FQDN name for your bucket, eg assets0.mydomain.com then you can create a CNAME record in your own DNS for your domain named assets0.mydomain.com and set the value to assets0.mydomain.com.s3.amazonaws.com and then load the assets directly from http://assets0.mydomain.com.

For a more detailed instruction set on Amazon S3 consult the Amazon documentation.

So where to start – well at first glance it seemed like it was going to be troublesome but it turns out it’s not that bad thanks to ActiveRecord. First up, we need to create a connection to a mySQL server – we’ll do this via database.yml (fortunately you can have multiple db connections per application);

# database.yml
legacy:
adapter: mysql
database:
host: 127.0.0.1
username:
password:

Then we essentially needed to ‘ActiveRecord-ify’ the legacy models coming from the mySQL database – fortunately ActiveRecord makes this super simple. So we created a Legacy:Base class which extended ActiveRecord base classes;

# legacy.rb
class Legacy::Base < ActiveRecord::Base
self.abstract_class = true
establish_connection "legacy"
end

The we simply created legacy models for each model in the mySQL database extending our legacy base class – the only thing these needed to know was the name of the table they corresponded to – just to make sure ActiveRecord was happy. (It turns out that our tables used an ‘id’ column akin to AR so didn’t need to explicitly set this on each model)

...
# legacy.rb
class Legacy::ForumCategory < Legacy::Base
set_table_name 'forum_categories'
#set_primary_key 'id'
end

This now meant we could do the following in script/console;

By extending the legacy base class which extended ActiveRecord our legacy models get full ActiveRecord support rolled into them – so it’s then super simple to iterate over the data and create a new model object for use in the new application, populate the object and then save it. And of course, since it’s ActiveRecord we don’t care what the target database is.

In the end, we created a rake task to create a repeatable process for importing the users from the DB – which also allowed us to handle any situations where the data we were trying to insert into our new models was failing validation.

Monitoring is done via an agent (written in python) installed on the server to give access to various standard metrics as well as stats exposed by mod_status on Apache. At the moment it’s only supported on Linux/OSX but they are working on a Windows agent.

Server Density provides both a web based dashboard as well as an iPhone application (free). The dashboard displays summary data as well as various graphs which reveal what’s going on on your server. Alerts can be configured against metrics and delivered via email, sms as well as iPhone notifications! My first iPhone application which uses notifications! Woohoo!

You’re able to trial the full service for 30 days, after which you can continue to monitor a single server but without the pleasure of the iPhone app, limited alerts and data is only retained for 1 month. After the trial period is over it’s going to cost you £10/$15 a month per server – a tad pricey I think though.

I’ve been using it for a few days now and already have caught 2 instances where my server was overloaded and using overly high amounts of resources – no real bother since I’m running on the Rackspace Cloud so just allocated more resources to my server.

The team behind it also have an excellent blog at http://blog.boxedice.com which has some *REALLY* insightful posts into how they outgrew Virtual Servers, moving from a relational DB to a non relational DB (MongoDB) and how they implemented iPhone push notifications – it’s well worth subscribing too!

If you’re in the market for server monitoring then it’s worth taking a look at!

As a result of this weeks FLOSS which had Eric Jung author of FoxyProxy on the show, he also spoke about his other tool PasswordMaker which really attracted my attention.

PasswordMaker does exactly what it says on the tin, makes passwords. It exists in a variety of formats, a Firefox extension, an OSX desktop widget, an online version (with code available as download) and many more! But where PasswordMaker differs is that it will generate you a complex password for each site/resource you visit and you don’t need to store them anywhere – you only need to remember one password – “One password to rule them all!”

It works by taking your master password along with the address of a website (usually just the domain bit or any other string) and generating you a complex password using a one way hash – it will generate the same password every time if you enter the same master password, account settings and address/text string so you don’t ever need to store the generated passwords, you simply regenerate them each time – and with the multitude of methods available to generate passwords it’s made really simple. There are many different encryption algorithms to choose from, password lengths, password characters so provided you use the same settings whenever you want to retrieve your password then you’ll always get the same password returned.

The Firefox extension makes it really simple – on clicking the icon you are presented with a screen to enter your master password, the ‘using text’ field will have been prefilled with the domain of the site (configurable to use subdomain, query string etc too via options) you are visiting. Once you enter your master password, the extension will generate a password which you can copy to the clipboard and use it from there – it even clears it after a set amount of time.

Should you need to generate a password for say MSN, then you can override the address with a simple string of text say ‘msnmessenger’ which will then be used to produce a password for you – equally server logins, FTP accounts, DB connections – provided you have a master password and a memorable unique identifier for the thing being secured then it will work for you – provided you can accept a minor inconvenience to have to enter your master password each time you need to use it.

It’s really worth checking out! Why not try it on a few sites and set how you get on…surely being safer on the internet is worth a little inconvenience?

ps If you have room for another podcast then add FLOSS to your pod catcher – it stands for Free Libre Open Source Software.

The guys also spoke about Mapstraction. This is REALLY interesting. Rather than coding your maps to a specific mapping API, mapstraction provides an API to 11 of the major mapping providers allowing you to switch providers at will! Granted you would need to wait for the mapstraction API to catch up with a providers API but removing lockin to a certain provider can only be a good thing. We have a number of sites implemented with different mapping providers so being able to use a common API would be damn handy!

Camera app bug seems to have been fixed – taking a picture now the taken image minimises to the bottom left corner as it should do.

The sync bug seems to be fixed too – previously after plugging in my iPhone into not my usual sync’ed MAC would cause the current play position to be reset to when it was last synced with master but now it does seem to retain the position correctly.

Update 2nd Sept – It turns out this bug is not fixed!!

Apps seem to load faster than on 3.0 rather than the slight pause that existed prior but it does screen that the pause on the black screen still occurs when you reboot the phone.

Any other fixes or new bugs introduced found?

In IIS when configuring a custom error page for say a 404 error you have the option of specifying a URL or a File (via an absolute path) to the friendly error page you wish to display.

As obvious as it seems now but if you specify a URL say /errors/404.htm then you will not get a 404 header returned by the webserver instead you get a 200 ok – which in hind sight makes total sense. To get a correct 404 header returned you need to use the file option and browse to the file you wish to display.