Category Archives: Uncategorized

Catalyst auto HTML/XSS scrubbing

At work, we needed to implement some HTML scrubbing and XSS protection across a Perl Catalyst-powered API, so went looking for existing solutions. We found Catalyst::Plugin::HTML::Scrubber which did some of what we needed, but did not scrub within encoded PUT/POST bodies e.g. POSTed JSON.

I implemented some improvements to provide this, but sadly the original author could not be reached – it seems he hasn’t been active in the Perl community for quite some time. With a little help from the CPAN admins (thanks!) I obtained maintainership of it, and have since got a couple of releases out which add the features we needed:

  • scrubbing HTML/XSS attempts within both normal parameters (querystring / POSTed forms) and also recursively within PUTted/POSTed JSON etc
  • the ability to whitelist certain parameters by name or regex to exclude them from scrubbing – we have some admin-only areas where staff can enter “message of the day” content which is allowed to contain HTML
  • a “no encode HTML entities” option to undo HTML::Scrubber‘s automatic HTML entity encoding of e.g. angle brackets – whilst content destined for the browser wants to be HTML-encoded, inbound parameters don’t want that, we just want HTML /XSS attempts stripped, a parameter value like >= 5 should be left alone

The amended version can be found on CPAN – Catalyst::Plugin::HTML::Scrubber.


(Aside: yes, it has been, er, quite some time since I posted anything on this blog.)

Why I love Perl : It makes things easy and fun

A short example of why I love Perl. The other day, I was picking my parents up from the airport; I do this reasonably regularly, as they go to visit family out in Ireland. Anyway, I’ve often wanted something to monitor the status of the incoming flight on Luton Airport’s website and tell me when it changes (if it’s delayed and a new estimated time is announced, or when it’s landed, etc). I looked around for decent services which do this, but found nothing which really appealed.

So, the other day, I decided “write it yourself, it’ll only take a few minutes”. Sure enough, with the aid of LWP::Simple, HTML::TableExtract, SMS::AQL and a little glue code, it was done:

#!/usr/bin/perl

use strict;
use LWP::Simple;
use HTML::TableExtract;
use SMS::AQL;

my $flight = shift or die "Usage: $0 flightnum";
my $url ="http://www.london-luton.co.uk/FlightData.ashx"
    . "?dir=arr&lang=en&id=1&r=20016807";
my $dest = '+44******';

my $sms = new SMS::AQL({
    username => '******',
    password => '******',
    options => { sender => '+44*****' },
});

my $last_status;
check:
while (1) {
    sleep 30 if $last_status;
    my $html = LWP::Simple::get($url) or warn "Failed to fetch HTML" and next;
    my $te = HTML::TableExtract->new(
        headers => [ 'Flight No', 'Airport', 'Scheduled', 'Flight Status' ]
    );
    $te->parse($html) or warn "Failed to parse HTML" and next;

    for my $row ($te->rows) {
        if ($row->[0] eq $flight) {
            if ($row->[3] ne $last_status) {
                $sms->send_sms($dest, "Flight $flight now $row->[3]");
            }
            $last_status = $row->[3];

            next check;
        }
    }
}

How easy was that?

(The code isn’t quite up to the quality I’d aim for if I was releasing it as an actual project – I’d tidy it up a bit, add more error checking/handling and have it read details from a conf file if I were to do that, but it’s a good example of what you can quickly rock up with the power of Perl and CPAN. I’d probably also swap it to SMS::Send, so it could be more easily used with various SMS gateways (I use www.aql.com – there’s an SMS::Send::AQL driver I wrote for that, which uses SMS::AQL under the hood anyway).)

SMS spam from 07873517718

Just got a random spammy text message from +447873517718 reading:

Records passed to us show you are entitled to £3750 for your accident for details reply CLAIM

Googling for the number shows nothing of use, so thought I’d whack a quick post up for the benefit of anyone else Googling. Now, I wonder if I can report it, and whether it’ll make any difference?

Apparently, you can forward spam text messages to 7726, for Orange, O2, T-Mobile and Three customers, or 87726 for Vodafone (who I’m with), according to a few sites, but I can’t see that being too useful; for one, forwarded text messages, by default, do not include the originating number.

Commenting now works again…

Whoops – when I migrated my blog from blog.preshweb.co.uk to www.preshweb.co.uk, commenting was broken because the keys I was using for reCAPTCHA’s API were no longer valid for the new hostname.

I’ve just changed the keys to ones which will work, so commenting should be fixed now. Thanks to Barbie for emailing me to let me know about it!

Blackberry Bold Christmas wallpapers (480×320)

I recently upgraded from a Blackberrry Curve to a Blackberry Bold 9000, which has a higher screen resolution (480×320), so I decided to re-do the wallpapers I resized & cropped and shared last year, and figured I may as well share the results.

Below are a selection of wallpapers I chose; some my own photos, the others are taken from various places around the web. I believe them to all be in the public domain but unfortunately I’m not certain, as I didn’t think to note where each one was downloaded from.

Each thumbnail links directly to the full-size 480×320 image, so you can just right-click & “Save Target As” or similar.

Enjoy!

SSH and rsync on Lacie Ethernet Disk Mini v2

Lacie Ethernet Disk Mini v2

A couple of months back, I acquired a Lacie Ethernet Disk Mini v2 for backups – I’m planning to give it to a friend to plug in, so I’ll have a little self-contained box to back up to.

The drive is a pretty stylish-looking and very solid-feeling device.

However, it comes with rather limited software – a rather poor and ugly web interface (even worse when you see the code behind it), and it supports Samba (SMB/CIFS) shares, FTP, some kind of Apple file sharing protocol, and uPnP media streaming.

Ignoring all that though, what I want is the ability to back up to it by rsync – something it doesn’t support, out of the box. However, it’s an ARM-powered unit running Linux underneath, so with a little trickery, you can make it much more functional.

There are several guides on uncrippling it to get SSH access to make proper use of it – I followed this one.

Basically, the process consists of opening up the device, and temporarily hooking up the hard drive directly to your computer in order to add a telnet binary and a backdoor to the web interface. I used a little USB to SATA/IDE adaptor I bought from eBuyer, meaning I could hook it up happily to my laptop, and didn’t have to remove the drive from the Lacie unit, simply removing the outer case and unplugging the cables from the drive and plugging in the ones from the USB unit.

See the guide linked to above for the full process, but you basically drop in a shell script which you can call via its web interface once it’s back up to execute whatever you want. The webserver on it runs at root (ugh) so it can start whatever you want it to. Typically, you’ll start utelnetd so that you can then telnet to it, download some ARM packages to install OpenSSH, rsync and other bits, get SSH working, then disable telnet, the horrible web interface, and anything else you don’t plan to use (I disabled proftpd, Samba, and the uPnP media sharing software).

Once it’s all done, you have a small but usable Linux system you can SSH to:

lacienas1

Naturally it’s not super-fast, but it does the job well enough!

A few tech specs, for anyone interested:

davidp@EDmini davidp $ cat /proc/cpuinfo 
Processor	: ARM926EJ-Sid(wb) rev 0 (v5l)
BogoMIPS	: 266.24
Features	: swp half thumb fastmult 
CPU implementer	: 0x41
CPU architecture: 5TEJ
CPU variant	: 0x0
CPU part	: 0x926
CPU revision	: 0
Cache type	: write-back
Cache clean	: cp15 c7 ops
Cache lockdown	: format C
Cache format	: Harvard
I size		: 32768
I assoc		: 1
I line length	: 32
I sets		: 1024
D size		: 32768
D assoc		: 1
D line length	: 32
D sets		: 1024

Hardware	: MV-88fxx81
Revision	: 0000
Serial		: 0000000000000000
davidp@EDmini davidp $ cat /proc/meminfo 
MemTotal:        61632 kB
MemFree:          1380 kB
Buffers:           188 kB
Cached:          28924 kB
SwapCached:          0 kB
Active:          17336 kB
Inactive:        23328 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:        61632 kB
LowFree:          1380 kB
SwapTotal:      128448 kB
SwapFree:       128448 kB
Dirty:            3472 kB
Writeback:           0 kB
Mapped:          14548 kB
Slab:            16688 kB
CommitLimit:    159264 kB
Committed_AS:    56104 kB
PageTables:        508 kB
VmallocTotal:   450560 kB
VmallocUsed:       716 kB
VmallocChunk:   449844 kB

“White-label” Nominet tag? Not quite…

I still receive occasional newsletters from Heart Internet, after trying out their free web hosting a long time ago (actually, I seem to recall it being a plan to mirror some content there, and never actually getting round to it).

Anyway, their latest newsletter sings the praises of their new “generic Nominet TAG”:

Previously, because of Nominet’s rules surrounding the registration of .uk domain names, we have had to register our reseller’s domain names through our Nominet tag, HEARTINTERNET, thus potentially exposing ourselves to reseller’s clients. Not any longer! We have now added an additional Nominet tag to our control panel for our resellers to use called EXTEND. This allows all our resellers to register domain names or transfer domain names through a generic tag, helping maintain their brand’s integrity.

I’m not exactly sure how this is helpful – instead of your clients seeing HEARTINTERNET they see EXTEND; it’s still not your brand, and Nominet’s tag list page still clearly identifies the tag as belonging to “Heart Internet Ltd t/a eXtend”.

(I was however amused by the wording of “potentially exposing ourselves to reseller’s clients” – that’s probably something best avoided! :) )