Category Archives: Perl

Perl, the duct tape of the Internet. Posts about Perl development, releases of my CPAN modules, etc.

Bot::BasicBot::Pluggable::Module::GitHub released

I’ve released a new distribution for Bot::BasicBot::Pluggable powered IRC bots, providing some useful GitHub-related functionality, named Bot::BasicBot::Pluggable::Module::GitHub.

The following modules are included – see the documentation for each for more details on how to use them.

Bot::BasicBot::Pluggable::Module::GitHub::EasyLinks

Provides quick URLs to view issues/pull requests etc when someone mentions one – for example:


<user> Go have a look at Issue 42
<bot1> Issue 42 (It doesn't work) https://github.com/....
<user> I fixed that in 5fcbb01
<bot1> Commit 5fcbb01 (Retarded logic fail.) - https://github.com/....

You can set a default project per-channel, so the above examples will look at whatever project is set as default for the channel the message was in.

You can also explicitly tell it to look at any other GitHub project:


<user> 5fcbb01 @ bigpresh/Bot-BasicBot-Pluggable-Module-GitHub
<bot1> Commit 5fcbb01 (Retarded logic fail.) - https://github.com/...

Bot::BasicBot::Pluggable::Module::GitHub::PullRequests

Monitor pull requests for GitHub projects.

Example:


<@bigpresh> !pr
< sophie> Open pull requests for sukria/Dancer : 8 pull requests open (felixdo:3, perlpilot:1, jamhed:1, dams:1, ambs:1, JTimothyKing:1)

Bot::BasicBot::Pluggable::Module::GitHub::Announce

Announces issues raised/closed for each channel’s default project.

Periodically checks on issues for each project, and reports changes,
for example:


< sophie> Issues closed : 667 (YAML::XS for Session::YAML) by jamhed : https://github.com/sukria/Dancer/issues/667

The code is on CPAN, and is available on GitHub. Contributions / bug reports / suggestions encouraged.

Using p3rl.org for FF search keyword

The lazy side of me enjoys Quick Search bookmarks in Firefox, so that I can type e.g. cpan ModuleName to look up a module on CPAN etc, pd perlfoo to look up the imaginary perlfoo documentation on perldoc, etc.

I also like p3rl.org‘s intelligent handling of short URLs, e.g. http://p3rl.org/Dancer, http://p3rl.org/perlrun etc.

It suddenly occurred to me to create a keyword search with the keyword p3, and the URL as http://p3rl.org/%s – now I can jump straight to pages I want with e.g. p3 Dancer, p3 perlrun etc.

My laziness likes this very much indeed.

Happy birthday Dancer!

Today marks two years to the day since the first version of Dancer hit CPAN!

According to the BackPAN, Dancer-0.9003.tar.gz hit CPAN on 07-Aug-2009.

I think you’ll agree we’ve come a long way since then, thanks to the awesome community and user base built up around the project since then.

In these two years, we’ve had countless valuable contributions from a large list of contributing users (see the list on the about page), gathered over 300 watchers on GitHub, had 84 people fork the repository on GitHub, had 620 pull requests submitted… amazing stuff.

Continue reading Happy birthday Dancer!

Renaming MP3s based on ID3 tags with Perl

As mentioned in my previous post on retagging MP3s by filename, I have a fairly large music collection, and prefer to keep it well organised.

I like the filenames used to always follow the same pattern, so I wrote mp3-rename, a Perl script to rename them based on the filename.

It takes a directory name as an argument (if not provided, it will operate upon the current directory). It will find all the MP3 files within that directory, extract information from the ID3 tags, then display a table showing the renames it will make if you tell it to go ahead, then ask for confirmation.

An example in action:


[davidp@supernova:~]$ mp3-rename /[...]/Rob\ Costlow\ -\ Woods\ of\ Chaos/
The following renames will be performed:
01 - Meant to Be.mp3    -> 00 - Meant to Be - Rob Costlow.mp3                                                           
02 - Reflections.mp3    -> 00 - Reflections - Rob Costlow.mp3                                                           
03 - Semester Days.mp3  -> 00 - Semester Days - Rob Costlow.mp3                                                         
[...]

Review the above renames to check the info is correct.

Type 'yes' to go ahead: ?>

(Some lines omitted for brevity)

It uses CPAN modules to do most of the work, including File::Find::Rule to find matching files to work on, Music::Tag to handle the tag processing, and Term::ReadKey to find out the terminal size.

Retagging MP3s by filename with Perl

I have a fairly large music collection in MP3s, and I like my files to be sensibly named and tagged.

One of the tools I use to do that is a Perl script I wrote named retag-by-filename, which allows you to provide a regular expression with named captures (so requires Perl 5.10 or later), and uses that to retag a bunch of MP3 files.

An example of it in use:


[davidp@supernova:~]$ ls -1 /shared/music/Complete\ Albums/Oasis\ -\ The\ Masterplan/ | head -3
01 - Acquiesce - Oasis.mp3
02 - Underneath The Sky - Oasis.mp3
03 - Talk Tonight - Oasis.mp3

[davidp@supernova:~]$ ./retag-by-filename --dry-run \
  --pattern="(?<track> \d+) \s - \s (?<title> .+ ) \s - \s (?<artist> .+ ) \.mp3" \
  /shared/music/Complete\ Albums/Oasis\ -\ The\ Masterplan/*.mp3
[01] Acquiesce by Oasis ()
[02] Underneath The Sky by Oasis ()
[03] Talk Tonight by Oasis ()

The --dry-run option shows the details of each track, but doesn’t actually update the tags. The --pattern option supplies the regular expression to match against filenames; you’ll use named captures named track, title, artist and comment to capture the appropriate parts. (In the example above, there is no comment to match.)

Thanks to the awesome power afforded by using modules from CPAN (Music::Tag and Getopt::Lucid), the actual script is 37 lines of code.

Backing up Google contacts/calendar etc

I recently followed a discussion on Twitter after @thomasmonopoly‘s Google account was blocked for unknown reasons, denying him access to the data he’d entrusted Google with.

I casually mentioned that entrusting Google to store all your data without having backups yourself is a bad idea, and @jmrowland enquired as to how you can back up stuff you create “in the cloud” (using Google).

I figured I should share the Perl script I’m currently using to back up my Google contacts, calendar and Google Reader subscriptions, so I’ve uploadedbackup-google-stuff on GitHub.

As I only wrote the script for my own use it’s quite basic and perhaps not particularly user-friendly, but I thought it only fair to share it; if people are interested in it, I may extend it somewhat. I’m fairly sure there are other “backup your Google account” solutions already out there already, though.

Of course, backing up mail from Gmail is trivial as they offer IMAP access; I don’t rely on Gmail so I’ve not bothered with that myself.

EDIT: I’ve seen Backupify recommended as a good solution for backing up your stuff from Google.

Easy profiling for Dancer apps

Today I released Dancer::Plugin::NYTProf, which provides easy profiling for Dancer apps, powered by the venerable Devel::NYTProf.

Using it is simple – load the plugin in your app:

use Dancer::Plugin::NYTProf;

Then, use your app as normal, then, to view the profile data, point your browser to /nytprof (e.g. http://localhost:3000/nytprof, and you’ll get a list of profile runs (each request is profiled individually):

Select the profile run you wish to view, and nytprofhtml will be invoked behind the scenes to generate the HTML reports, which will then be served up, and you’ll be looking at the helpful Devel::NYTProf reports, to see where time was spent processing your request:

Early days yet, and a lot of room for improvement, but in my testing, it works.

Things I’d like to add when time permits:

  • The ability to exclude Dancer internals from the profiling (if I can find a clean way to do so)
  • The ability to enable profiling only for certain requests – for instance, providing a pattern to match the request URLs you want to profile
  • The ability to customise the URL at which profiling reports are served up
  • Check for sane behaviour if prefixes are in use

Feedback welcome!

Writing a Bugzilla extension to auto-link commits

I’ve been meaning to write a Bugzilla extension to turn mentions of commits in bug messages into a link to view the commit via our web-based SVN viewer for ages – this morning I finally found the time to do it.

I needed to use the bug_format_comment hook to format comments as they’re being displayed, turning mentions of commits (e.g. “Commit 123” or “r123”) into links.

The code was pretty simple:
Continue reading Writing a Bugzilla extension to auto-link commits

Secure password handling in Dancer apps with Bcrypt

James Aitken has released Dancer::Plugin::Bcrypt, a new plugin for Dancer apps to make secure password hashing using Bcrypt easy.

For a background on why you ought to use Bcrypt rather than simpler hashing, see http://codahale.com/how-to-safely-store-a-password/ – basically, using MD5/SHA etc is too inexpensive, meaning that, even with a good salt, cracking the hash isn’t too hard to do these days, especially with the advent of use of the GPU. Bcrypt is intentionally expensive and slow (you can decide just how much).

Dancer::Plugin::Bcrypt makes validating a password hash as easy as:


if (bcrypt_validate_password($entered_password, $stored_hash)) {
    ...
}

Generating a hash to store is also very simple:


my $hash = bcrypt($plaintext);

Generation of random salt is taken care of for you.

*UPDATE* – the above is a nice simple way to quickly get secure password hashing with minimal effort – it is likely not the best way, though. If you’re already using DBIx::Class, then see DBIx::Class::PassphraseColumn for a better way to do this automatically at your database model level.

Thanks to mst for prompting me to mention the above :)