Category Archives: Geeky

Technical stuff about Perl, Linux and computing & technology in general.

Perl, Programming

Backing up Google contacts/calendar etc

6 Comments

I recently followed a discussion on Twitter after @thomasmonopoly‘s Google account was blocked for unknown reasons, denying him access to the data he’d entrusted Google with.

I casually mentioned that entrusting Google to store all your data without having backups yourself is a bad idea, and @jmrowland enquired as to how you can back up stuff you create “in the cloud” (using Google).

I figured I should share the Perl script I’m currently using to back up my Google contacts, calendar and Google Reader subscriptions, so I’ve uploadedbackup-google-stuff on GitHub.

As I only wrote the script for my own use it’s quite basic and perhaps not particularly user-friendly, but I thought it only fair to share it; if people are interested in it, I may extend it somewhat. I’m fairly sure there are other “backup your Google account” solutions already out there already, though.

Of course, backing up mail from Gmail is trivial as they offer IMAP access; I don’t rely on Gmail so I’ve not bothered with that myself.

EDIT: I’ve seen Backupify recommended as a good solution for backing up your stuff from Google.

Dancer web framework, Perl, Programming

Easy profiling for Dancer apps

2 Comments

Today I released Dancer::Plugin::NYTProf, which provides easy profiling for Dancer apps, powered by the venerable Devel::NYTProf.

Using it is simple – load the plugin in your app:

use Dancer::Plugin::NYTProf;

Then, use your app as normal, then, to view the profile data, point your browser to /nytprof (e.g. http://localhost:3000/nytprof, and you’ll get a list of profile runs (each request is profiled individually):

Select the profile run you wish to view, and nytprofhtml will be invoked behind the scenes to generate the HTML reports, which will then be served up, and you’ll be looking at the helpful Devel::NYTProf reports, to see where time was spent processing your request:

Early days yet, and a lot of room for improvement, but in my testing, it works.

Things I’d like to add when time permits:

  • The ability to exclude Dancer internals from the profiling (if I can find a clean way to do so)
  • The ability to enable profiling only for certain requests – for instance, providing a pattern to match the request URLs you want to profile
  • The ability to customise the URL at which profiling reports are served up
  • Check for sane behaviour if prefixes are in use

Feedback welcome!

Perl

Writing a Bugzilla extension to auto-link commits

2 Comments

I’ve been meaning to write a Bugzilla extension to turn mentions of commits in bug messages into a link to view the commit via our web-based SVN viewer for ages – this morning I finally found the time to do it.

I needed to use the bug_format_comment hook to format comments as they’re being displayed, turning mentions of commits (e.g. “Commit 123” or “r123”) into links.

The code was pretty simple:
Continue reading Writing a Bugzilla extension to auto-link commits →

Dancer web framework, Perl

Secure password handling in Dancer apps with Bcrypt

James Aitken has released Dancer::Plugin::Bcrypt, a new plugin for Dancer apps to make secure password hashing using Bcrypt easy.

For a background on why you ought to use Bcrypt rather than simpler hashing, see http://codahale.com/how-to-safely-store-a-password/ – basically, using MD5/SHA etc is too inexpensive, meaning that, even with a good salt, cracking the hash isn’t too hard to do these days, especially with the advent of use of the GPU. Bcrypt is intentionally expensive and slow (you can decide just how much).

Dancer::Plugin::Bcrypt makes validating a password hash as easy as:


if (bcrypt_validate_password($entered_password, $stored_hash)) {
    ...
}

Generating a hash to store is also very simple:


my $hash = bcrypt($plaintext);

Generation of random salt is taken care of for you.

*UPDATE* – the above is a nice simple way to quickly get secure password hashing with minimal effort – it is likely not the best way, though. If you’re already using DBIx::Class, then see DBIx::Class::PassphraseColumn for a better way to do this automatically at your database model level.

Thanks to mst for prompting me to mention the above :)

Geeky, Linux, System Administration

SMART monitoring drives on 3ware cards

I’ve been updating my Nagios monitoring to make sure I’m carefully monitoring my server hardware, including temperature, fan speeds and hard drives, and I wanted to use S.M.A.R.T. monitoring to monitor drives on a 3ware RAID controller for signs of imminent failure.

I already monitor the status of the RAID array itself using my nagios_3ware_raid_check Nagios plugin (which I previously blogged about), but I wanted to use SMART monitoring to look for signs of imminent drive trouble, rather than simply finding out when a drive has just failed.

After installing smartmontools, I was able to edit /etc/smartd.conf, disabling the default of scanning for devices, and listing devices explicitly, as follows:


# Monitor the drives on our RAID array; schedule self tests for Sundays.
/dev/twa0 -d 3ware,0 -a -s L/../../7/02
/dev/twa0 -d 3ware,1 -a -s L/../../7/04

The above monitors both drives of a RAID-1 mirrored pair on a 3ware controller card; the -s option schedules a long self-test every Sunday, starting between 2-3am and 4-5am respectively.

I’m still looking for a good way to monitor via Nagios, though; the (poorly-named) check_ide_smart plugin doesn’t support the ability to monitor drives on other interfaces as far as I can see. I found a couple of Perl scripts such as Check-SMART-status-modified, but they had issues.

Geeky

Using Yahoo Pipes to strip images from RSS feed

I’ve been subscribed to the It Made My Day RSS feed for ages, but fairly recently they started adding pointless cutesy image macros as well as the text – I don’t want to see those in the feed as they’re generally hard to read and a waste of bandwidth.

So, Yahoo Pipes to the rescue – I created a simple pipe which fetches the IMMD feed, uses regexes to strip out the images, then subscribed to the result of that pipe.

Job done – if you’re interested, the pipe feed URL to subscribe to is http://pipes.yahoo.com/bigpresh/immdnoimages.