PreshBlog

Well, December is upon us – time for advent calendars, and as usual, the Perl community doesn’t disappoint – here’s a list of the Perl-related advent calendars I’m aware of:

• Dancer Advent Calendar – the Dancer web framework’s calendar (feed)
• Perl Advent Calendar (perladvent.org) (feed)
• Catalyst Advent Calendar – the Catalyst web framework’s calendar (feed)
• Perl6 Advent Calendar (feed)
• SysAdvent – an advent calendar for sysadmins, but often Perl-related

There are also several Japanese-language advent calendars:

• http://perl-users.jp/articles/advent-calendar/2011/:
  • AnySan Track
  • Casual Track
  • dbix Track
  • English Track
  • Hacker Track
  • Test Track
  • Acme Track
  • Teng Track
  • Amon2 Track

If you know of any others, please feel free to let me know and I’ll add them to the list :)

Yesterday, I attended the 2011 London Perl Workshop – my first ever Perl conference.

I had a good day, met a few members of the Perl community I knew from online interactions who I’d never met in person before, saw some good talks, and partook in some free food and beer (kindly paid for by the sponsors, including my employer, UK2).

Some brief mentions of talks I attended:

Matt S Trout (mst) – First, Tak wrote the world‎

I have IO::Pipeline. I have App::FatPacker. I have IPC::Command::Multiplex. And yet I still couldn’t whip up a five line example of bolting them all together that made a compelling argument for a perl-loving sysadmin to stop using fabric.

This problem, among others, will be solved by the conclusion of this talk.

Tak sounds like something which will be very useful to me – running code on multiple other hosts via SSH, but including Perl code – with all locally-installed modules available for use at the remote end!

As mst went through explaining how it all worked, my thoughts went from “hmm, useful”, to “hmm, useful but looks over-engineered, not sure it needs to be that complex” to “whoah, that’s genius”. Fatpacking and sending code to the remote side, which then adds a coderef to @INC which requests other modules from the local end, sent over and loaded remotely, is awesomely creative.

These kind of tricks remind me of why I love Perl.

Mike Whitaker (‎Penfold‎) – ‎Perl and Unicode, the 5.14 edition‎

A very good talk on handling Unicode safely in Perl, and the gotchas to avoid. Provided major impetus for me to upgrade to 5.14, too.

Zefram – ‎why time is difficult‎

Dates, times, time intervals, clocks, calendars, and related phenomena are major contributors to hassle in programming, and the source of innumerable bugs.

Zefram’s talk, whilst barely Perl related, was very interesting, and very well delivered. I hadn’t realised quite how complex time was :)

Zefram’s amusing lightning talk on doing away with source code by simply storing bytecode and editing it by deparsing the source, editing it, then “compiling” back to bytecode was also entertaining.

Claes Jakobsson (‎claes‎) – ‎Don’t debug now, debug later

Runops::Recorder is a alternate runloop for perl that writes down what your program does to disk for playback later

It also comes with a viewer and some helper classes for you to write your own playback tools such as diffs etc.

This looks like a very useful debugging tool, recording the path of execution through your code and writing it to a file which can then later be “replayed” using a viewer – much like single-stepping or tracing through the debugger, but after the fact. The ability to leave it running and have it dump out a configurable amount of trace data when a die is encountered looks excellently useful for catching intermittent / rare problems – you should be able to leave it in place, wait until the problem occurs, then replay what happened leading up to the die to see what was going on.

Future versions should also be able to track changes to variables, etc, which will be very useful indeed.

There were a couple of workshops I’d like to have attended, but which I didn’t; partly because they conflicted with talks I wanted to see, and partly because I didn’t have a laptop with me to “work along” and didn’t think I could take much of value away from them.

Andrew Solomon – ‎[[TRAINING SESSION]] Web development for beginners using Dancer‎

As a core developer for the Dancer perl web framework I’d love to have attended Andrew Solomon’s workshop, to see what was being taught, and offer any input desired. Unfortunately, I wasn’t there, but I’ll be looking with interest for any feedback from people who were, and what they learned and what they thought of Dancer if they hadn’t encountered it before. Making Perl accessible for new users is an important thing.

Gabor Szabo (‎szabgab‎) – ‎[[TRAINING SESSION]] Testing in Perl

I’d also like to have taken part in Gabor’s workshops, but they were in two parts and conflicted with several other talks I wanted to see.

I met a few members of the Perl community who I knew from online interactions but had never met in meatspace, so it was great to meet them. Unfortunately, there were a few others I’d meant to go introduce myself to, but never got a chance to do so – including Tatsuhiko Miyagawa and Gabor Szgabo.

Overall, it was a good day, and I imagine there’s a very good chance I’ll be back next year :)

I recently wanted to produce some graphs from a web app powered by the Dancer Perl web framework, and reevaluated the various Perl graphing moduiles out there.

Modules I considered were:

• Chart::Strip
• Chart::Graph
• Google::Chart
• Chart::Clicker
• Chart::Gnuplot

Unfortunately, I didn’t have time to do a full in-depth writeup trying every module like the excellent ones Neil Bowers has been doing, but I thought I’d write up a quick post on the choice I made, with example code, in case it helps other people looking to graph potentially irregularly-spaced time-based data samples in Perl easily.

Chart::Clicker looked to be a nice choice (with a nice example of doing just what I want given as the topic answer to a question on StackOverflow), but had a huge chain of dependencies, finally failing when demanding Cairo and various X11 libraries (on my headless server).

Chart::Strip seemed to do exactly what I wanted in a simple way, but I encountered a div-by-zero bug when dealing with a certain dataset with > 89 data points.

I reported this to the author, Jeff Weisgberg in RT #72288, and he promptly released 1.08 with a fix (thanks Jeff!).

Chart::Strip made it simple to do what I wanted:

my @dataset;
while (my $row = $sth->fetchrow_hashref) {
    push @dataset, { time => $row->{timestamp}, value => $row->{value} };
}
 
my $chart = Chart::Strip->new( title => "My chart" );
$chart->add_data(\@dataset, { style => 'line' });
 
# then get the chart as an image with $chart->png

Nice and easy, just what I wanted – a way to say “here’s some timestamps and values (quite possibly irregularly spaced) – work out how to plot this sensibly for me”.

The resulting graphs look good enough to me, e.g.:





(Rendered intentionally a little smaller to fit the blog; naturally the graphs can be whatever size you want. Also, I had to use the transparent option to disable transparent backgrounds.)

IRC (Internet Relay Chat) is a protocol for multi-server text chat between many participants in many channels, started back in 1988.

There are plenty of IRC networks out there for social chatter, including the likes of Freenode and irc.perl.org hosting many channels for Perl and open source channels in general, making it easy to get quick help from developers and users of your favourite project.

However, I find IRC to be a very valuable tool indeed to help development teams collaborate effectively; at work we make extensive use of it. It’s useful whether you’re a formal development team in a corporate environment, or an open source project whose developers / collaborators gather on IRC.

Why is it so useful? Well:

It enables quick discussion and collaboration without breaking your workflow

As a developer, you don’t want to lose your concentration – when you’re “in the zone”, you’re carrying information about the code you’re working on in your brain, and it doesn’t stay there for long if you’re distracted. Someone walking over to you and starting talking to you, or a phone call, demand more or less 100% attention; you will be distracted, and you will “fall out of the zone”, causing your productivity to fall until you get back to where you were.

IRC, on the other hand, means you don’t have to respond quite so immediately, and I find it easy to flick between coding and IRC (both terminals within Terminator for me) without losing focus on where I’m up to and what I’m doing.

Most IRC clients support alerting you when your nick is mentioned in a channel or you receive a direct message, so you can ignore general chatter in the channel until you’re ready to read it, but know if someone is trying to get your attention.

Of course, it’s even more valuable when your development team work from multiple locations, whether that’s having employees working from home, or multiple offices.

Logs of discussions can be valuable for future reference

If you keep logs of your discussions, it’s easy to refer back to later – sometimes you’ll remember “ah, yes, we talked about this – what was the outcome?” – quick log search, and your answer is there. “Why did we decide that this was the best way to implement this?” – log search – “ah, that was why”.

Open, widely-supported protocol

IRC is an open, widely supported protocol; there’s various clients available for pretty much every platform, so whatever system your devs work on, they’ll be able to find a client that suits them.

Easily extensible to integrate with other tools

It’s easy to write IRC “bots” which can help integrate with various other tools in various ways.

A good example is providing easy links to commits / bug reports or issues / pull requests etc.

If you’re using my Bot::BasicBot::Pluggable::Module::GitHub for instance, you can mention an issue and have the bot automatically provide a summary and an URL for anyone who wants to see what the issue in question is – e.g.:

<user1> Anyone had a moment to look at Issue 42 and see what's going on?
<bot> Issue 42 (It doesn't work) https://github.com/....
<user2> Oh yeah - I fixed that in 5fcbb01
<bot> Commit 5fcbb01 (Retarded logic fail.) - https://github.com/....

It’s easy to cobble together a simple bot or bot module to do this kind of stuff for whatever your in-house situation requires, if there’s nothing suitable already out there on CPAN (which, a lot of the time, there already will be).

GitHub provide post-receive hooks which can be configured to announce pushes to your IRC channel(s). Bot::BasicBot::Pluggable::GitHub::Announce can automatically announce new/updated issues, and, in future releases, also pull requests and commits/pushes.

System problems reported instantly

Use something like my Bot::BasicBot::Pluggable::Module::Nagios, and you can have system problems reported automatically to the appropriate IRC channels, for quick attention by whoever needs to deal with them. I use an applet in my GNOME system tray which alerts me to problems, but seeing them reported in detail on IRC is handy, and also strikes up conversation about it – a simple “I’m on it ^^” is enough to let others know you’re dealing with the issue and they don’t need to worry about it.

Announce tweets about your company/brand/project/interests

My Bot::BasicBot::Pluggable::Module::TwitterWatch module allows you to have the bot watch for and report new posts on Twitter about your company/project/brand/stuff of interest, and post them to your IRC channel – either for awareness, or to strike up discussion about them.

Wrap various other tools

Your IRC bot(s) can provide various other useful facilities – for instance, find the corelist command useful? Bot::BasicBot::Pluggable::Module::CoreList makes it easy for your bot to answer corelist lookups within the flow of a conversation.

<user1> Could use File::Spec - that's part of core, isn't it?
<user2> bot: corelist File::Spec
<bot> File::Spec was first released with perl 5.00503 (released on 1999-03-28)
<user2> Yep :)

I’ve just released Bot::BasicBot::Pluggable::Module::Nagios – a module for IRC bots powered by Bot::BasicBot::Pluggable which monitors one or more Nagios instances and reports problems to IRC channels.

I’ll be using this at work to have service problems reported to us on IRC for quick attention, but figured it’s something that’s likely to be of use to others elsewhere, too, so I’m releasing it.

There’s still some more features and improvements I want to make (a TODO list is included in the module POD), but it’s at a state where I consider it to be usable (it works for me).

Feedback/suggestions welcome.

It should be on a CPAN mirror near you soon, and the repo is on GitHub should you wish to submit pull requests or raise issues for bug reports/feature requests.

I’ve released a new distribution for Bot::BasicBot::Pluggable powered IRC bots, providing some useful GitHub-related functionality, named Bot::BasicBot::Pluggable::Module::GitHub.

The following modules are included – see the documentation for each for more details on how to use them.

Bot::BasicBot::Pluggable::Module::GitHub::EasyLinks

Provides quick URLs to view issues/pull requests etc when someone mentions one – for example:

<user> Go have a look at Issue 42
<bot1> Issue 42 (It doesn't work) https://github.com/....
<user> I fixed that in 5fcbb01
<bot1> Commit 5fcbb01 (Retarded logic fail.) - https://github.com/....

You can set a default project per-channel, so the above examples will look at whatever project is set as default for the channel the message was in.

You can also explicitly tell it to look at any other GitHub project:

<user> 5fcbb01 @ bigpresh/Bot-BasicBot-Pluggable-Module-GitHub
<bot1> Commit 5fcbb01 (Retarded logic fail.) - https://github.com/...

Bot::BasicBot::Pluggable::Module::GitHub::PullRequests

Monitor pull requests for GitHub projects.

Example:

<@bigpresh> !pr
< sophie> Open pull requests for sukria/Dancer : 8 pull requests open (felixdo:3, perlpilot:1, jamhed:1, dams:1, ambs:1, JTimothyKing:1)

Bot::BasicBot::Pluggable::Module::GitHub::Announce

Announces issues raised/closed for each channel’s default project.

Periodically checks on issues for each project, and reports changes,
for example:

< sophie> Issues closed : 667 (YAML::XS for Session::YAML) by jamhed : https://github.com/sukria/Dancer/issues/667

The code is on CPAN, and is available on GitHub. Contributions / bug reports / suggestions encouraged.

Version 1.30801.3072 of the Dancer Perl web framework was released today.

It was codenamed the "Precious David Precious" release after me – thanks guys :)

The changes included are:
(continue reading…)

Today marks two years to the day since the first version of Dancer hit CPAN!

According to the BackPAN, Dancer-0.9003.tar.gz hit CPAN on 07-Aug-2009.

I think you’ll agree we’ve come a long way since then, thanks to the awesome community and user base built up around the project since then.

In these two years, we’ve had countless valuable contributions from a large list of contributing users (see the list on the about page), gathered over 300 watchers on GitHub, had 84 people fork the repository on GitHub, had 620 pull requests submitted… amazing stuff.

(continue reading…)

As mentioned in my previous post on retagging MP3s by filename, I have a fairly large music collection, and prefer to keep it well organised.

I like the filenames used to always follow the same pattern, so I wrote mp3-rename, a Perl script to rename them based on the filename.

It takes a directory name as an argument (if not provided, it will operate upon the current directory). It will find all the MP3 files within that directory, extract information from the ID3 tags, then display a table showing the renames it will make if you tell it to go ahead, then ask for confirmation.

An example in action:

[davidp@supernova:~]$ mp3-rename /[...]/Rob\ Costlow\ -\ Woods\ of\ Chaos/
The following renames will be performed:
01 - Meant to Be.mp3    -> 00 - Meant to Be - Rob Costlow.mp3
02 - Reflections.mp3    -> 00 - Reflections - Rob Costlow.mp3
03 - Semester Days.mp3  -> 00 - Semester Days - Rob Costlow.mp3
[...]

Review the above renames to check the info is correct.

Type 'yes' to go ahead: ?>

(Some lines omitted for brevity)

It uses CPAN modules to do most of the work, including File::Find::Rule to find matching files to work on, Music::Tag to handle the tag processing, and Term::ReadKey to find out the terminal size.

I have a fairly large music collection in MP3s, and I like my files to be sensibly named and tagged.

One of the tools I use to do that is a Perl script I wrote named retag-by-filename, which allows you to provide a regular expression with named captures (so requires Perl 5.10 or later), and uses that to retag a bunch of MP3 files.

An example of it in use:

[davidp@supernova:~]$ ls -1 /shared/music/Complete\ Albums/Oasis\ -\ The\ Masterplan/ | head -3
01 - Acquiesce - Oasis.mp3
02 - Underneath The Sky - Oasis.mp3
03 - Talk Tonight - Oasis.mp3

[davidp@supernova:~]$ ./retag-by-filename --dry-run \
  --pattern="(?<track> \d+) \s - \s (?<title> .+ ) \s - \s (?<artist> .+ ) \.mp3" \
  /shared/music/Complete\ Albums/Oasis\ -\ The\ Masterplan/*.mp3
[01] Acquiesce by Oasis ()
[02] Underneath The Sky by Oasis ()
[03] Talk Tonight by Oasis ()

The --dry-run option shows the details of each track, but doesn’t actually update the tags. The --pattern option supplies the regular expression to match against filenames; you’ll use named captures named track, title, artist and comment to capture the appropriate parts. (In the example above, there is no comment to match.)

Thanks to the awesome power afforded by using modules from CPAN (Music::Tag and Getopt::Lucid), the actual script is 37 lines of code.